debug-inference
Debug inference clients that use an attached provider and its native endpoint, including hosted APIs and host-local Ollama, vLLM, SGLang, TRT-LLM, LM Studio, or NIM. Use for provider attachment, endpoint policy, credential substitution, topology, and migration from the removed managed inference endpoint. Trigger keywords - debug inference, managed inference endpoint, local inference, ollama, lm studio, vllm, sglang, trtllm, NIM, inference failing, model server unreachable, credentialendpointmismatch, host.openshell.internal.
- Installs
- 473
- Last 24h
- +21
- Trending rank
- #5693
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/nvidia/openshell --skill debug-inference- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads debug-inference automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Debug Inference
Diagnose inference as ordinary provider-authorized network traffic. OpenShell no longer supplies a managed inference route, rewrites request shapes, or selects a model. The application calls the provider's native endpoint and owns its base URL, model, request format, and timeout.
Use installed openshell --help output as the authority for command syntax.
Refer to the published provider management guide
and provider profile guide
for current behavior.
Diagnostic Workflow
1. Confirm Gateway and Sandbox Context
openshell status
openshell gateway info
openshell sandbox get <sandbox>
For a host-local model server, host.openshell.internal identifies the machine
running the gateway. It does not identify the operator's laptop when the gateway
is remote. A server listening only on 127.0.0.1 may also be unreachable from a
container; bind it to an address reachable from the gateway runtime.
2. Inspect the Provider and Its Profile
openshell provider get <provider>
openshell profile export <profile-id> -o yaml
Check that the profile:
- Names the exact endpoint host, port, and protocol the client calls.
- Allows the client binary.
- Declares the credential key and intended authentication style.
- Uses narrow HTTP rules when the provider should expose only part of an API.
For a custom or self-hosted OpenAI-compatible endpoint, import an endpoint-bearing profile. A base URL stored only in provider configuration does not authorize a new endpoint.
openshell profile lint -f ./provider-profile.yaml
openshell profile import -f ./provider-profile.yaml
openshell provider create --name <provider> --type <profile-id>
Add the required --credential KEY or --credential KEY=VALUE arguments shown
by the profile. Never broaden endpoint policy merely to silence a credential
binding error.
3. Confirm Attachment
openshell sandbox provider list <sandbox>
openshell sandbox provider attach <sandbox> <provider> --wait --timeout 30
Save the change's receipt_id and use openshell sandbox provider status <sandbox> <provider> --receipt <receipt-id> --wait --timeout 30 to check when it takes effect. Success confirms that the sandbox applied the credentials, policy, and environment for new processes. If the result is pending, failed, withheld, or superseded, inspect its reason before launching the client.
Launch the client after the attachment wait succeeds so it receives the updated environment:
openshell sandbox exec <sandbox> -- <client-command>
After updating an ordinary static provider, wait for that change and launch a new client. An existing process keeps its revision-scoped reference; readiness does not make the old reference resolve the replacement value. Diagnose managed-refresh credentials according to their own lifecycle.
Keep credentials and issued references out of diagnostic output. Acknowledged detach revokes future credential resolution and removes the reference from future process environments. Requests already forwarded may still finish:
openshell sandbox provider detach <sandbox> <provider> --wait --timeout 30
4. Verify Native Client Configuration
The application must use the real upstream contract:
- Native provider base URL, not the retired managed virtual endpoint.
- Real model ID, not a placeholder that OpenShell used to rewrite.
- Native OpenAI, Anthropic, Vertex, or other provider request shape.
- Application-owned timeout and retry settings.
- The credential environment variable declared by the attached profile.
Probe the exact endpoint from a newly launched sandbox process. Start with a non-secret discovery endpoint when the provider offers one, then send a minimal inference request using the provider's documented API shape.
5. Interpret Common Failures
| Symptom | Likely cause | Fix |
|---|---|---|
credential_placeholder_in_request_body | A body reference is invalid/revoked, or classification metadata is unavailable | Check the controlled denial reason; remove the reference from conversation history or restore provider access. Do not enable body credential rewriting or bypass flags to send tool output. Unknown literals and valid issued placeholders pass unchanged, including the model provider’s own placeholder. Header resolution does not enable body rewriting. |
| A retired managed endpoint fails DNS resolution | Client still uses the removed managed endpoint | Configure the provider's native base URL and attach an endpoint-bearing provider profile |
| Direct request is denied | Missing attachment, endpoint policy, HTTP rule, or binary authorization | Inspect the attached provider profile and sandbox effective policy |
credential_endpoint_mismatch | Credential profile does not authorize the request recipient | Correct the host/port/path or import a narrowly scoped profile for the intended endpoint |
request_authority_mismatch | HTTP authority differs from the CONNECT destination | Use the same host and effective port in both authorities |
| Credential variable is absent | Provider was not attached when this process launched, or profiles collide on a key | Attach the provider and launch a new process; resolve duplicate keys explicitly |
| Upstream rejects the model or body | Client relied on removed model/request rewriting | Configure the real model and provider-native request format in the application |
127.0.0.1 works on the host but not in the sandbox | Loopback refers to different runtime | Use host.openshell.internal or another gateway-reachable endpoint and profile |
| Host-local request times out | Server bind address, gateway topology, or host firewall blocks container-to-host traffic | Verify the listener and permit only the required gateway network path and port |
Host-Local Inference Checklist
For Ollama, LM Studio, vLLM, SGLang, TRT-LLM, and local NIM deployments:
- Verify the engine from the gateway host.
- Verify it listens on an address reachable from the gateway runtime.
- Import a custom profile naming
host.openshell.internaland the actual port. - Restrict the profile to the intended binaries and API paths.
- Create and attach the provider.
- Configure the application's base URL, model, and timeout.
- Probe the native endpoint from a newly launched sandbox process.
Reporting
Report:
- The active gateway and whether topology contributes to the failure.
- The provider, profile, attachment, endpoint, and client binary involved.
- The exact failed host, port, path, and request authority without secrets.
- Whether the client still relies on removed managed-routing behavior.
- The narrowest profile, attachment, or application configuration change that resolves the problem.
FAQ
What does the debug-inference skill do?
Debug inference clients that use an attached provider and its native endpoint, including hosted APIs and host-local Ollama, vLLM, SGLang, TRT-LLM, LM Studio, or NIM. Use for provider attachment, endpoint policy, credential substitution, topology, and migration from the removed managed inference endpoint. Trigger keywords - debug inference, managed inference endpoint, local inference, ollama, lm studio, vllm, sglang, trtllm, NIM, inference failing, model server unreachable, credentialendpointmismatch, host.openshell.internal.
How do I install debug-inference?
Run npx skills add https://github.com/nvidia/openshell --skill debug-inference in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/debug-inference/ (personal) or .claude/skills/debug-inference/ (project).
Which AI agents can use debug-inference?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is debug-inference safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains debug-inference?
It is published by nvidia in nvidia/openshell, and it is an official skill from the maker of the technology it covers. It has 473 recorded installs.
More from nvidia/openshell
openshell-cli
nvidia/openshell
Guide agents through using the OpenShell CLI (openshell) for sandbox management, gateway registration, provider configuration and refresh, profile management, policy iteration, settings, service exposure, BYOC workflows, and attached-provider inference. Covers basic through advanced multi-step workflows. Trigger keywords - openshell, sandbox create, sandbox exec, sandbox connect, logs, provider create, profile list, profile describe, provider refresh, policy set, policy get, settings, service expose, forward, port forward, BYOC, bring your own container, inference, use openshell, run openshell, CLI usage, manage sandbox, manage provider, gateway add, gateway select.
debug-openshell-cluster
nvidia/openshell
Debug why an OpenShell gateway deployment is unhealthy, unreachable, or unable to create sandboxes. Use for gateway health failures, Docker/Podman runtime issues, Helm failures, Kubernetes scheduling, TLS or auth, gateway interceptors, supervisor middleware startup or runtime failures, external compute-driver sockets, VM drivers, or sandbox startup. Trigger keywords - debug gateway, gateway failing, deployment failing, helm install failing, cluster health, gateway health, gateway not starting, health check failed, sandbox pending, docker driver, podman driver, kubernetes driver, external driver, compute driver socket, gateway interceptor, supervisor middleware, middleware failed, vm driver.
generate-sandbox-policy
nvidia/openshell
Generate sandbox security policies from plain-language requirements and optional REST API documentation. Produces L4 or fine-grained L7 network policies and ordered network middleware configuration. Use for API access rules, middleware host selection, failure behavior, or built-in and operator-run middleware attachment. Trigger keywords - generate policy, create policy, update policy, change policy, sandbox policy, network policy, API policy, security policy, allow API, restrict API, network middleware, supervisor middleware.
Popular ai & agents skills
google-agents-cli-publish
google/agents-cli
This skill should be used when the user wants to "publish an agent", "publish my ADK agent", "register an agent with Gemini", "publish to Gemini" (formerly Gemini Enterprise), or needs guidance on the agents-cli publish gemini command. Also use when the user wants to "manage agents in Agent Registry", "list/update/delete registered agents", or "register an MCP server". Covers ADK vs A2A registration modes, programmatic and interactive usage, flag reference, auto-detection from deployment metadata, Agent Registry fleet management, and troubleshooting. Part of the agents-cli skills suite. Do NOT use for deployment (use google-agents-cli-deploy).
google-agents-cli-eval
google/agents-cli
This skill should be used when the user wants to "run an evaluation", "evaluate my agent", "evaluate my ADK agent", "write an eval dataset", "analyze eval failures", "compare eval results", "optimize agent", or needs guidance on the Gemini platform eval methodology and the Quality Flywheel. Covers eval metrics, dataset schema, LLM-as-judge scoring, and common failure causes. Applies to any agents-cli project, whatever framework the agent is written in. Do NOT use for agent API code patterns (ADK: use google-agents-cli-adk-code), deployment (use google-agents-cli-deploy), or project scaffolding (use google-agents-cli-scaffold).
skill-creator
anthropics/skills
Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.
codex-pet
prime-skills/runcomfy-agent-skills
Codex Pet generator on RunComfy. Build a Codex-compatible Codex Pet spritesheet.webp + pet.json from a single reference image, drop it into ${CODEXHOME:-$HOME/.codex}/pets/ / and Codex picks it up as a custom Codex Pet next to the 8 built-ins. This skill produces the exact Codex Pet atlas Codex expects (1536x1872 PNG/WebP, 8 cols x 9 rows, 192x208 cells, 9 animation states — idle, running-right, running-left, waving, jumping, failed, waiting, running, review). Calls OpenAI GPT Image 2 edit ONCE via the local RunComfy CLI as runcomfy run openai/gpt-image-2/edit to produce a canonical Codex Pet pose, then assembles all 9 animation rows programmatically with ImageMagick micro-transforms — no Codex Pro, no $imagegen, no OPENAIAPIKEY required, only RUNCOMFYTOKEN. Triggers on "codex pet", "create codex pet", "make codex pet", "hatch codex pet", "/hatch image", "desktop pet codex", "codex pets", "spritesheet.webp", or any explicit ask to build a custom pet for OpenAI Codex.
full-output-enforcement
leonxlnx/taste-skill
Overrides default LLM truncation behavior. Enforces complete code generation, bans placeholder patterns, and handles token-limit splits cleanly. Apply to any task requiring exhaustive, unabridged output.
executing-plans
obra/superpowers
Use when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available