audit-agent-history
Mine the user's own agent session logs per model and per harness, count failure modes, and propose the top three as hazard rules. Use when the user asks to audit, review or mine agent history, session logs or transcripts for mistakes, failure modes or corrections, or asks why an agent keeps doing X.
- Installs
- 563
- Last 24h
- +22
- Trending rank
- #5356
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/timschoch/skilly --skill audit-agent-history- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads audit-agent-history automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Audit agent history
Rules written from guesses miss. Rules written from counted corrections in the user's own logs hit. This skill counts, then proposes at most three lines for bundles/workflow/rules/workflow-hazards.md, plus the consumer overlay pairs worth hoisting into their hub rule.
Where the logs live
Verify each path with ls first. Skip what is absent, say so in the report.
| Harness | Path | Shape |
|---|---|---|
| Claude Code | ~/.claude/projects/<project-slug>/*.jsonl | one file per session, one JSON message per line; type is user or assistant; cwd names the repo; assistant lines carry model (claude-fable-5, claude-opus-5) |
| Codex | ~/.codex/sessions/<yyyy>/<mm>/<dd>/rollout-*.jsonl | one file per session; first line type: session_meta with cwd; later lines carry model (gpt-5.6) |
The project slug is the repo path with / replaced by -: /Users/x/repo/skilly becomes -Users-x-repo-skilly.
Scope
Ask for, or default:
- repos: which
cwdvalues; default all - window: default 30 days, by file mtime or
timestamp - models: default all seen
Run the whole audit in one subagent. It is deep reasoning over many logs. Raw logs never enter the main context, see principle-guard-the-context-window. The subagent returns the report only.
Method
- Count sessions and user messages per model and harness. Both are denominators for step 4.
- Find corrections: a user message that follows an assistant action and reverses it. Match on
no,don't,stop,I said,why did you,undo,revert,not what I asked,I didn't ask. Read the two messages before each hit to drop false positives (anothat answers a question is not a correction). - Classify each correction into one mode from references/failure-modes.md. One mode per correction. Unmatched goes to
otherwith a two-word label. - Count per model per mode. Normalise:
corrections / user messages * 100. Raw counts favour the model used most. - Quote two real examples for each of the top three modes: the assistant action and the user's correction. Redact secrets and every path under the home directory (
~/...). - Interrogate one bad thread. Open the worst session in its own agent with the same model and ask: what gave the indication this was right, which line in the rule file or
CLAUDE.mdwas outdated, where was the request misread first. Record the answer under the mode it explains. - Interrogate one slow thread. Take the session with the most tool calls per user message. Group its tool calls into categories (orientation reads, repeated reads, checks, edits, verification). Mark the groups that changed nothing about the result as useless.
- Collect overlay pairs. In each repo in scope, read
.claude/rules/*.local.md. These hold the bad/good pairs consumers wrote under a synced rule, see.claude/rules/workflow-writing-standards.md. A pair that appears in two or more repos, or matches a top-three mode, is a hoist candidate for the rule it extends.
Output
Report table, one row per model and mode, sorted by per-100 descending:
| mode | model | count | per 100 messages | example |
Then a ## Proposed hazards block. At most three rules in the format of bundles/workflow/rules/workflow-hazards.md: one numbered line, Never plus the action, then the reason. Each rule carries one bad and one good example from the logs.
1. Never kill a process by port number alone. Two harness instances share a port range; the audit found four kills of the running session in two days. Bad: `kill $(lsof -t -i:3000)`. Good: `ps -o pid,command | grep <name>`, then kill the one pid.
Then a ## Hoist candidates block: one line per overlay pair, naming the repo, the rule file and item it extends, and the pair verbatim.
Then hand-off:
- Show the report and both blocks to the user. Stop.
- On sign-off, edit the rule files under
bundles/workflow/rules/in the hub repo (timschoch/skilly) only. Never in a consumer repo: the rule syncs from the hub and the next sync overwrites a local edit. Hoisted pairs go under the item they extend; the consumer then deletes them from its.local.md.
References
- references/failure-modes.md: the modes, the log signal for each, one example
FAQ
What does the audit-agent-history skill do?
Mine the user's own agent session logs per model and per harness, count failure modes, and propose the top three as hazard rules. Use when the user asks to audit, review or mine agent history, session logs or transcripts for mistakes, failure modes or corrections, or asks why an agent keeps doing X.
How do I install audit-agent-history?
Run npx skills add https://github.com/timschoch/skilly --skill audit-agent-history in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/audit-agent-history/ (personal) or .claude/skills/audit-agent-history/ (project).
Which AI agents can use audit-agent-history?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is audit-agent-history safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains audit-agent-history?
It is published by timschoch in timschoch/skilly. It has 563 recorded installs.
More from timschoch/skilly
git-shortcuts
timschoch/skilly
Translates shorthand git/GitHub workflow commands into the correct sequence of actions. ALWAYS trigger when the user's message contains a "--" prefixed shorthand like: "--c", "--p", "--cp", "--pr", "--prm", "--cppr", "--cpprm", "--ff", "--release", "--pub", "--close" — with or without a trailing "?". The "--" prefix is required — do NOT trigger on bare letters like "c" or "pr" appearing in normal sentences. NEVER invoke this skill autonomously — it MUST always be user-invoked via an explicit "--" shorthand in their message.
cleanup-merged-branches
timschoch/skilly
Delete remote branches whose PR was merged. Read-only dry-run, then confirm, then delete.
step-by-step
timschoch/skilly
Walk the user through a manual procedure step by step, with every value spelled out and every link clickable.
plan-critics
timschoch/skilly
Critiques an implementation plan — or a wayfinder map/ticket — from multiple angles (security, simplicity, edge cases, scope creep, codebase conflicts, tech-specific best practices) via parallel critic subagents. Trigger: user says "kritisiere den plan", "plan check", "plan-kritik", "blinde flecken", "/plan-critics" — also invoke when a planning agent wants to validate its own plan before presenting it, or a wayfinder session wants a resolution checked before closing a ticket. Must be called at top-level (not inside a subagent) — it uses the Task tool.
qa
timschoch/skilly
Interactive QA session where user reports bugs or issues conversationally, and the agent files GitHub issues. Explores the codebase in the background for context and domain language. Use when user wants to report bugs, do QA, file issues conversationally, or mentions "QA session".
1-3-1
timschoch/skilly
Frame a problem or decision as 1-3-1: one problem, three distinct solutions, one recommendation. Use when the user brings a non-trivial decision or fork and wants options weighed before a pick ("what should I do about X", "help me decide", "weigh the tradeoffs"), or names "1-3-1" or "do a 131". Skip questions with one right answer, just answer those.
Popular security skills
azure-compliance
microsoft/azure-skills
Run Azure compliance and security audits with azqr plus Key Vault expiration checks. Covers best-practice assessment, resource review, policy/compliance validation, and security posture checks. WHEN: compliance scan, security audit, BEFORE running azqr (compliance cli tool), Azure best practices, Key Vault expiration check, expired certificates, expiring secrets, orphaned resources, compliance assessment.
minimal-run-and-audit
lllllllama/rigorpilot-skills
Rigor Run skill for README-first deep learning repo reproduction. Use when the task is specifically to capture or normalize evidence from the selected smoke test or documented inference or evaluation command and write standardized reprooutputs/ files, including patch notes when repository files changed. Do not use for training execution, initial repo intake, generic environment setup, paper lookup, target selection, hidden scientific-meaning changes, or end-to-end orchestration by itself.
seo-audit
coreyhaines31/marketingskills
When the user wants to audit, review, or diagnose SEO issues on their site. Also use when the user mentions "SEO audit," "technical SEO," "why am I not ranking," "SEO issues," "on-page SEO," "meta tags review," "SEO health check," "my traffic dropped," "lost rankings," "not showing up in Google," "site isn't ranking," "Google update hit me," "page speed," "core web vitals," "crawl errors," "indexing issues," "stuck on page 2," "striking distance keywords," "title tag rewrite," "low CTR," "local SEO," "Google Business Profile," or "map pack." Use this even if the user just says something vague like "my SEO is bad" or "help with SEO" — start with an audit. For building pages at scale to target keywords, see programmatic-seo. For adding structured data, see schema. For AI search optimization, see ai-seo.
firebase-security-rules-auditor
firebase/agent-skills
Audits Firebase (Firestore, Cloud Storage) security rules for vulnerabilities, privilege escalation, role bypasses, create vs update inconsistencies, resource exhaustion, type safety, size limits, and hasOnly ownership checks. Use when auditing/reviewing rules, running red-team rule assessments, or scoring against auditor checklists. Don't use for Firebase CLI (login, deploy), Auth, Crashlytics, Remote Config, or database queries.
audit
pbakaus/impeccable
Run technical quality checks across accessibility, performance, theming, responsive design, and anti-patterns. Generates a scored report with P0-P3 severity ratings and actionable plan. Use when the user wants an accessibility check, performance audit, or technical quality review.
competitors
coreyhaines31/marketingskills
When the user wants to create competitor comparison or alternative pages for SEO and buyer-facing use. Also use when the user mentions 'alternative page,' 'vs page,' 'competitor comparison,' 'comparison page,' '[Product] vs [Product],' '[Product] alternative,' 'competitive landing pages,' 'how do we compare to X,' 'competitor teardown,' 'audit our competitor pages,' 'are our comparison pages out of date,' or 'competitive asset audit.' Use this for any content that positions your product against competitors. Covers four formats: singular alternative, plural alternatives, you vs competitor, and competitor vs competitor. For auditing existing claims (not researching competitors from scratch, which is competitor-profiling; not technical SEO on these pages, which is seo-audit), use the asset audit here. For internal battle cards and sales-specific competitor docs, see sales-enablement.