loop-diagnosis
Diagnose and fix stalled agent loops using the joelclaw CLI. Use when loops appear stuck, stories aren't progressing, or the event chain broke. Triggers on "loop stalled", "why isn't the loop progressing", "diagnose loops", "fix stuck loop", "loop not moving", "what happened to the loop", "stories stuck at pending", or any request to debug loop infrastructure.
- Installs
- 39
- Last 24h
- —
- Trending rank
- —
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/joelhooks/joelclaw --skill loop-diagnosis- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads loop-diagnosis automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Loop Diagnosis
Diagnose and fix stalled agent coding loops. This skill covers the diagnostic CLI, common failure modes, and the observability patterns that prevent silent stalls.
Quick Commands
# Diagnose all active loops at once
joelclaw loop diagnose all -c
# Diagnose a specific loop
joelclaw loop diagnose <loop-id> -c
# Diagnose AND auto-fix
joelclaw loop diagnose all -c --fix
# Full JSON output (for detailed inspection)
joelclaw loop diagnose <loop-id>
What Diagnosis Checks
The diagnose command runs 6 checks in order:
- Redis state — PRD stories (pass/skip/pending), progress entries, active claims
- Worktree — exists? commits? uncommitted changes? .out files?
- Inngest runs — running/failed agent-loop-* functions, recent plan runs
- Agent processes — any claude/codex processes still alive?
- Worker health — function_count from localhost:3111/api/inngest
- Diagnosis — pattern-matches the above into a root cause
Failure Modes & Fixes
| Diagnosis | Root Cause | Auto-Fix |
|---|---|---|
CHAIN_BROKEN | Judge sent story.passed but plan never received it. Event lost in transit. | Re-fires agent/loop.story.passed → plan picks next story |
ORPHANED_CLAIM | Story claimed by an event, but agent died and no Inngest run is active. | Clears claim + re-fires plan event |
STUCK_RUN | Inngest run marked RUNNING but agent process is dead. Run won't complete. | Clears claims + re-fires (manual run cancellation may be needed in Inngest dashboard) |
WORKER_UNHEALTHY | Worker registering fewer functions than expected. Missing imports or crash loop. | Restarts system-bus-worker deployment in k8s |
NO_PRD | Loop has no PRD in Redis — was nuked or never created. | None — start a new loop |
COMPLETE | All stories passed or skipped. Nothing to do. | None — run joelclaw loop nuke dead to clean up |
When to Use (vs Other Skills)
- loop-diagnosis → Loop is stuck/stalled, need to figure out why and fix it
- loop-nanny → Loop is running, need to monitor progress and clean up after
- agent-loop → Need to START a new loop
The Event Chain
Understanding the chain helps diagnose WHERE it broke:
agent/loop.started
→ plan (picks story, dispatches test-writer)
→ agent/loop.story.dispatched
→ test-writer (writes acceptance tests)
→ agent/loop.tests.written
→ implement (codex/claude writes code)
→ agent/loop.story.implemented
→ review (runs tests, typecheck, claude review)
→ agent/loop.story.reviewed
→ judge (pass/fail/retry decision)
→ agent/loop.story.passed ←── feeds back to plan
→ agent/loop.story.failed ←── feeds back to plan
→ agent/loop.story.retry ←── feeds back to implement
Most common break point: judge → plan. The agent/loop.story.passed event fires but plan never picks it up. This happens when:
- Inngest is restarting during the event
- Worker was restarted between judge and plan
- k8s pod restart dropped the event
Observability Patterns
Passive: Failure Events
Every loop function should emit failure events via onFailure handlers. These fire agent/loop.function.failed and must be visible through canonical OTel/Inngest event surfaces.
Active: Watchdog (Future)
A periodic Inngest function (system/loop-watchdog) that:
- Scans all loops in Redis with pending stories
- Checks if any events were emitted in the last 10 minutes
- If not → auto-runs diagnose + fix
- Emits OTel and writes a durable Brain receipt only when behavior or operator action changed
Manual: The Diagnostic Session
When an agent needs to debug loops manually, follow this sequence:
# 1. Quick overview
joelclaw loop diagnose all -c
# 2. If fix needed
joelclaw loop diagnose all -c --fix
# 3. Verify fix worked (wait ~30s for plan to fire)
joelclaw loop status <loop-id> -c
# 4. If still stuck, check worker
curl -s localhost:3111/api/inngest | python3 -c "import json,sys; print(json.load(sys.stdin)['function_count'])"
# 5. Nuclear option: full restart
joelclaw loop restart <loop-id>
Making Loops More Resilient
The root cause of most stalls is lost events in the judge→plan chain. Solutions being implemented:
- onFailure handlers — every function gets one, logs failure + emits diagnostic event
- Loop watchdog — periodic check for silent stalls
- Debounce on content-sync — prevents event storms that can crowd out loop events
- Singleton on backfill — prevents resource contention during loops
Cross-References
- agent-loop skill — starting loops
- loop-nanny skill — monitoring + cleanup
- joelclaw skill — full CLI reference
- ADR-0028 — reliability patterns
FAQ
What does the loop-diagnosis skill do?
Diagnose and fix stalled agent loops using the joelclaw CLI. Use when loops appear stuck, stories aren't progressing, or the event chain broke. Triggers on "loop stalled", "why isn't the loop progressing", "diagnose loops", "fix stuck loop", "loop not moving", "what happened to the loop", "stories stuck at pending", or any request to debug loop infrastructure.
How do I install loop-diagnosis?
Run npx skills add https://github.com/joelhooks/joelclaw --skill loop-diagnosis in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/loop-diagnosis/ (personal) or .claude/skills/loop-diagnosis/ (project).
Which AI agents can use loop-diagnosis?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is loop-diagnosis safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains loop-diagnosis?
It is published by joelhooks in joelhooks/joelclaw. It has 39 recorded installs.
More from joelhooks/joelclaw
contacts
joelhooks/joelclaw
Add, enrich, and manage contacts in Joel's Vault. Fire the Inngest enrichment pipeline for full multi-source dossiers, or create quick contacts manually. Use when: 'add a contact', 'enrich this person', 'who is X', 'VIP contact', 'update contact', or any task involving the Vault/Contacts directory.
cli-design
joelhooks/joelclaw
Design and build agent-first CLIs with HATEOAS JSON responses, context-protecting output, and self-documenting command trees. Use when creating new CLI tools, adding commands to existing CLIs such as joelclaw, or reviewing CLI design for agent-friendliness. Triggers on 'build a CLI', 'add a command', 'CLI design', 'agent-friendly output', or any task involving command-line tool creation.
docker-sandbox
joelhooks/joelclaw
Create, manage, and execute agent tools (claude, codex) inside Docker sandboxes for isolated code execution. Use when running agent loops, spawning tool subprocesses, or any task requiring process isolation. Triggers on "sandbox", "isolated execution", "docker sandbox", "safe agent execution", or when working on agent loop infrastructure.
granola
joelhooks/joelclaw
Access and process Granola meeting notes and transcripts via the granola CLI (MCP-backed). Use when pulling meeting data, analyzing transcripts, backfilling meetings, or any task involving Granola meeting content.
k8s
joelhooks/joelclaw
Operate the joelclaw Kubernetes cluster — Talos Linux on Colima (Mac Mini). Deploy services, check health, debug pods, recover from restarts, add ports, manage Helm releases, inspect logs, fix networking. Triggers on: 'kubectl', 'pods', 'deploy to k8s', 'cluster health', 'restart pod', 'helm install', 'talosctl', 'colima', 'nodeport', 'flannel', 'port mapping', 'k8s down', 'cluster not working', 'add a port', 'PVC', 'storage', any k8s/Talos/Colima infrastructure task. Also triggers on service-specific deploy: 'deploy redis', 'redeploy inngest', 'livekit helm', 'pds not responding'.
joel-writing-style
joelhooks/joelclaw
Joel's writing voice and style guide for joelclaw.com content. Use when writing, editing, or reviewing any blog post, essay, book chapter, or prose content for joelclaw.com. Also use when asked to 'write like Joel,' 'match Joel's voice,' 'draft a post,' 'write content for the blog,' or 'review this for voice.' This skill captures Joel's specific writing patterns derived from 90,000 words of published content spanning 2012–2026. Cross-reference with copy-editing and copywriting skills for marketing-specific copy.
Popular devops & cloud skills
microsoft-foundry
microsoft/azure-skills
Build, deploy, evaluate, optimize, fine-tune, and manage Microsoft Foundry agents, models, and resources end to end. USE FOR: foundry, azd ai agent, azd provision/deploy, hosted agent scaffold/develop/run/deploy/troubleshoot, prompt agent create, managed harness agent create, GitHub Copilot harness agent, create agent, update agent, add tool to agent, invoke agent, agent insights, evaluate agent, batch eval, continuous eval, continuous monitoring, agent CI/CD, optimize prompt, improve prompt, optimize agent instructions, Agent Optimizer scaffold, dataset curation from traces, deploy model, model fine-tuning (SFT/DPO/RFT), Foundry project, RBAC, role assignment, permissions, quota, capacity, region, deployment failure, AI Services, create Foundry resource, knowledge index, customize deployment, onboard, availability, training-data, grader, distillation, large file upload. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure
azure-diagnostics
microsoft/azure-skills
Debug Azure production issues on Azure using AppLens, Azure Monitor, resource health, and safe triage. WHEN: debug production issues, troubleshoot app service, app service high CPU, app service deployment failure, troubleshoot container apps, container app 503 or target port mismatch, troubleshoot functions, function telemetry missing or stopped after a config/connection-string change, troubleshoot AKS, VM RDP, Linux SSH, VM black screen, can't connect to VM, reset VM password, NSG or firewall blocking, kubectl cannot connect, kube-system/CoreDNS failures, pod pending, crashloop, node not ready, upgrade failures, analyze logs, KQL, insights, image pull failures, cold start issues, health probe failures, resource health, root cause of errors, troubleshoot event hubs, troubleshoot service bus, messaging SDK error, AMQP connection failure, message lock lost, lock renewal during long-running processing, service bus dead letter.
azure-prepare
microsoft/azure-skills
Prepare azd-based Azure projects for deployment: generates azure.yaml, infrastructure (Bicep/Terraform), and Dockerfiles for the Azure Developer CLI (azd) workflow. USE ONLY when the user explicitly wants to use azd as the deployment tool, or the project already has an azure.yaml file. DO NOT USE FOR: non-azd deployments, Python App Service code-only deploys (use python-appservice-deploy), or cross-cloud migration (use azure-cloud-migrate). WHEN: prepare app for azd, create azure.yaml, set up azd infrastructure, modernize app for Azure with azd, deploy with azd, function app, timer trigger, service bus trigger, event-driven function, managed identity, generate Bicep, generate Terraform, create and deploy to Azure.
azure-ai
microsoft/azure-skills
Use for Azure AI: Search, Speech, OpenAI, Document Intelligence. Helps with search, vector/hybrid search, speech-to-text, text-to-speech, transcription, OCR. WHEN: AI Search, query search, vector search, hybrid search, semantic search, speech-to-text, text-to-speech, transcribe, OCR, convert text to speech.
azure-deploy
microsoft/azure-skills
Execute Azure deployments for ALREADY-PREPARED applications that have existing .azure/deployment-plan.md and infrastructure files. DO NOT use this skill when the user asks to CREATE a new application — use azure-prepare instead. This skill runs azd up, azd deploy, terraform apply, and az deployment commands with built-in error recovery. Requires .azure/deployment-plan.md from azure-prepare and validated status from azure-validate. WHEN: "run azd up", "run azd deploy", "execute deployment", "push to production", "push to cloud", "go live", "ship it", "bicep deploy", "terraform apply", "publish to Azure", "launch on Azure". DO NOT USE WHEN: "create and deploy", "build and deploy", "create a new app", "set up infrastructure", "create and deploy to Azure using Terraform" — use azure-prepare for these.
azure-validate
microsoft/azure-skills
Pre-deployment validation for Azure readiness. Run deep checks on configuration, infrastructure (Bicep or Terraform), RBAC role assignments, managed identity permissions, and prerequisites before deploying. WHEN: validate my app, check deployment readiness, run preflight checks, verify configuration, check if ready to deploy, validate azure.yaml, validate Bicep, test before deploying, troubleshoot deployment errors, validate Azure Functions, validate function app, validate serverless deployment, verify RBAC roles, check role assignments, review managed identity permissions, what-if analysis, validate Container Apps deployment.