langsmith-trace
INVOKE THIS SKILL when working with LangSmith tracing OR querying traces. Covers adding tracing to applications and querying/exporting trace data. Uses the langsmith CLI tool.
- Installs
- 64
- Last 24h
- —
- Trending rank
- —
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/langchain-ai/skills-benchmarks --skill langsmith-trace- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads langsmith-trace automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
LANGSMITH_API_KEY=lsv2_pt_your_api_key_here # REQUIRED
LANGSMITH_PROJECT=your-project-name # Optional: default project
LANGSMITH_WORKSPACE_ID=your-workspace-id # Optional: for org-scoped keys
Authentication is REQUIRED: either set the LANGSMITH_API_KEY environment variable, or pass the --api-key flag to CLI commands (preferred):
langsmith trace list --project my-project --api-key $LANGSMITH_API_KEY
IMPORTANT: Always check the environment variables or .env file for LANGSMITH_PROJECT before querying or interacting with LangSmith. This tells you which project contains the relevant traces and data. If the LangSmith project is not available, use your best judgement to identify the right one.
CLI Tool
curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh
<trace_langchain_oss> For LangChain/LangGraph apps, tracing is automatic. Just set environment variables:
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=<your-api-key>
export OPENAI_API_KEY=<your-openai-api-key> # or your LLM provider's key
Optional variables:
LANGSMITH_PROJECT- specify project name (defaults to "default")LANGCHAIN_CALLBACKS_BACKGROUND=false- use for serverless to ensure traces complete before function exit (Python) </trace_langchain_oss>
<trace_other_frameworks>
For anything other than LangChain/LangGraph, read the matching reference file in references/ before writing tracing code. Each reference covers install, env vars, setup snippet, and gotchas specific to that framework. The setup is rarely identical across frameworks — picking the wrong pattern (e.g. using @traceable when the framework has native OTel) creates duplicate/missing spans.
Decision order:
- Framework has a dedicated reference below → use it
- Framework has native OpenTelemetry but no dedicated reference →
references/otel.md - No framework, or unsupported framework →
references/traceable.md - Cannot run a LangSmith SDK at all →
references/api.md(last resort)
Routing table:
| If you're tracing… | Read |
|---|---|
| OpenAI / Azure OpenAI / Anthropic / any plain LLM client | references/traceable.md |
| AutoGen | references/autogen.md |
| CrewAI | references/crewai.md |
| Google ADK | references/google-adk.md |
Google Gemini (google-genai SDK directly) | references/google-gemini.md |
| Instructor (structured outputs) | references/instructor.md |
| LiveKit Agents (voice AI) | references/livekit.md |
| Mastra (TypeScript) | references/mastra.md |
| Microsoft Agent Framework | references/microsoft-agent-framework.md |
| Mistral | references/mistral.md |
| n8n (self-hosted) | references/n8n.md |
| OpenAI Agents SDK | references/openai-agents-sdk.md |
| OpenCode | references/opencode.md |
| OpenAI Codex CLI | references/codex.md |
| Pipecat (voice AI) | references/pipecat.md |
| PydanticAI | references/pydantic-ai.md |
| Semantic Kernel | references/semantic-kernel.md |
| Strands Agents | references/strands-agents.md |
| Temporal workflows (Go/Python/TS) | references/temporal.md |
| Vercel AI SDK | references/vercel-ai-sdk.md |
| Any other framework with native OTel | references/otel.md |
| Multi-backend OTel fan-out | references/otel.md (Collector section) |
| Raw REST (no SDK available) | references/api.md |
If the framework you need isn't listed here, check references/ — new integrations are added there, not inline.
</trace_other_frameworks>
<traces_vs_runs>
Use the langsmith CLI to query trace data.
Understanding the difference is critical:
- Trace = A complete execution tree (root run + all child runs). A trace represents one full agent invocation with all its LLM calls, tool calls, and nested operations.
- Run = A single node in the tree (one LLM call, one tool call, etc.)
Generally, query traces first — they provide complete context and preserve hierarchy needed for trajectory analysis and dataset generation. </traces_vs_runs>
<command_structure> Two command groups with consistent behavior:
langsmith
├── trace (operations on trace trees - USE THIS FIRST)
│ ├── list - List traces (filters apply to root run)
│ ├── get - Get single trace with full hierarchy
│ └── export - Export traces to JSONL files (one file per trace)
│
├── run (operations on individual runs - for specific analysis)
│ ├── list - List runs (flat, filters apply to any run)
│ ├── get - Get single run
│ └── export - Export runs to single JSONL file (flat)
│
├── dataset (dataset operations)
│ ├── list - List datasets
│ ├── get - Get dataset details
│ ├── create - Create empty dataset
│ ├── delete - Delete dataset
│ ├── export - Export dataset to file
│ └── upload - Upload local JSON as dataset
│
├── example (example operations)
│ ├── list - List examples in a dataset
│ ├── create - Add example to a dataset
│ └── delete - Delete an example
│
├── evaluator (evaluator operations)
│ ├── list - List evaluators
│ ├── upload - Upload evaluator
│ └── delete - Delete evaluator
│
├── experiment (experiment operations)
│ ├── list - List experiments
│ └── get - Get experiment results
│
├── thread (thread operations)
│ ├── list - List conversation threads
│ └── get - Get thread details
│
└── project (project operations)
└── list - List tracing projects
Key differences:
traces * | runs * | |
|---|---|---|
| Filters apply to | Root run only | Any matching run |
--run-type | Not available | Available |
| Returns | Full hierarchy | Flat list |
| Export output | Directory (one file/trace) | Single file |
| </command_structure> |
<querying_traces>
Query traces using the langsmith CLI. Commands are language-agnostic.
# List recent traces (most common operation)
langsmith trace list --limit 10 --project my-project --api-key $LANGSMITH_API_KEY
# List traces with metadata (timing, tokens, costs)
langsmith trace list --limit 10 --include-metadata --api-key $LANGSMITH_API_KEY
# Filter traces by time
langsmith trace list --last-n-minutes 60 --api-key $LANGSMITH_API_KEY
langsmith trace list --since 2025-01-20T10:00:00Z --api-key $LANGSMITH_API_KEY
# Get specific trace with full hierarchy
langsmith trace get <trace-id> --api-key $LANGSMITH_API_KEY
# List traces and show hierarchy inline
langsmith trace list --limit 5 --show-hierarchy --api-key $LANGSMITH_API_KEY
# Export traces to JSONL (one file per trace, includes all runs)
langsmith trace export ./traces --limit 20 --full --api-key $LANGSMITH_API_KEY
# Filter traces by performance
langsmith trace list --min-latency 5.0 --limit 10 --api-key $LANGSMITH_API_KEY # Slow traces (>= 5s)
langsmith trace list --error --last-n-minutes 60 --api-key $LANGSMITH_API_KEY # Failed traces
# List specific run types (flat list)
langsmith run list --run-type llm --limit 20 --api-key $LANGSMITH_API_KEY
</querying_traces>
<filters> All commands support these filters (all AND together):Basic filters:
--trace-ids abc,def- Filter to specific traces--limit N- Max results--project NAME- Project name--last-n-minutes N- Time filter--since TIMESTAMP- Time filter (ISO format)--error / --no-error- Error status--name PATTERN- Name contains (case-insensitive)
Performance filters:
--min-latency SECONDS- Minimum latency (e.g.,5for >= 5s)--max-latency SECONDS- Maximum latency--min-tokens N- Minimum total tokens--tags tag1,tag2- Has any of these tags
Advanced filter:
--filter QUERY- Raw LangSmith filter query for complex cases (feedback, metadata, etc.)
# Filter traces by feedback score using raw LangSmith query
langsmith trace list --filter 'and(eq(feedback_key, "correctness"), gte(feedback_score, 0.8))' --api-key $LANGSMITH_API_KEY
<export_format>
Export creates .jsonl files (one run per line) with these fields:
{"run_id": "...", "trace_id": "...", "name": "...", "run_type": "...", "parent_run_id": "...", "inputs": {...}, "outputs": {...}}
Use --include-io or --full to include inputs/outputs (required for dataset generation).
</export_format>
FAQ
What does the langsmith-trace skill do?
INVOKE THIS SKILL when working with LangSmith tracing OR querying traces. Covers adding tracing to applications and querying/exporting trace data. Uses the langsmith CLI tool.
How do I install langsmith-trace?
Run npx skills add https://github.com/langchain-ai/skills-benchmarks --skill langsmith-trace in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/langsmith-trace/ (personal) or .claude/skills/langsmith-trace/ (project).
Which AI agents can use langsmith-trace?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is langsmith-trace safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains langsmith-trace?
It is published by langchain-ai in langchain-ai/skills-benchmarks, and it is an official skill from the maker of the technology it covers. It has 64 recorded installs.
More from langchain-ai/skills-benchmarks
react-components
langchain-ai/skills-benchmarks
Modern React component patterns with hooks and TypeScript
langgraph-fundamentals
langchain-ai/skills-benchmarks
Open the skill to read its SKILL.md instructions.
langchain-oss-primer
langchain-ai/skills-benchmarks
Open the skill to read its SKILL.md instructions.
langchain-dependencies
langchain-ai/skills-benchmarks
Open the skill to read its SKILL.md instructions.
framework-selection
langchain-ai/skills-benchmarks
Open the skill to read its SKILL.md instructions.
deep-agents-core
langchain-ai/skills-benchmarks
Open the skill to read its SKILL.md instructions.
Popular ai & agents skills
google-agents-cli-publish
google/agents-cli
This skill should be used when the user wants to "publish an agent", "publish my ADK agent", "register an agent with Gemini", "publish to Gemini" (formerly Gemini Enterprise), or needs guidance on the agents-cli publish gemini command. Also use when the user wants to "manage agents in Agent Registry", "list/update/delete registered agents", or "register an MCP server". Covers ADK vs A2A registration modes, programmatic and interactive usage, flag reference, auto-detection from deployment metadata, Agent Registry fleet management, and troubleshooting. Part of the agents-cli skills suite. Do NOT use for deployment (use google-agents-cli-deploy).
google-agents-cli-eval
google/agents-cli
This skill should be used when the user wants to "run an evaluation", "evaluate my agent", "evaluate my ADK agent", "write an eval dataset", "analyze eval failures", "compare eval results", "optimize agent", or needs guidance on the Gemini platform eval methodology and the Quality Flywheel. Covers eval metrics, dataset schema, LLM-as-judge scoring, and common failure causes. Applies to any agents-cli project, whatever framework the agent is written in. Do NOT use for agent API code patterns (ADK: use google-agents-cli-adk-code), deployment (use google-agents-cli-deploy), or project scaffolding (use google-agents-cli-scaffold).
skill-creator
anthropics/skills
Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.
codex-pet
prime-skills/runcomfy-agent-skills
Codex Pet generator on RunComfy. Build a Codex-compatible Codex Pet spritesheet.webp + pet.json from a single reference image, drop it into ${CODEXHOME:-$HOME/.codex}/pets/ / and Codex picks it up as a custom Codex Pet next to the 8 built-ins. This skill produces the exact Codex Pet atlas Codex expects (1536x1872 PNG/WebP, 8 cols x 9 rows, 192x208 cells, 9 animation states — idle, running-right, running-left, waving, jumping, failed, waiting, running, review). Calls OpenAI GPT Image 2 edit ONCE via the local RunComfy CLI as runcomfy run openai/gpt-image-2/edit to produce a canonical Codex Pet pose, then assembles all 9 animation rows programmatically with ImageMagick micro-transforms — no Codex Pro, no $imagegen, no OPENAIAPIKEY required, only RUNCOMFYTOKEN. Triggers on "codex pet", "create codex pet", "make codex pet", "hatch codex pet", "/hatch image", "desktop pet codex", "codex pets", "spritesheet.webp", or any explicit ask to build a custom pet for OpenAI Codex.
full-output-enforcement
leonxlnx/taste-skill
Overrides default LLM truncation behavior. Enforces complete code generation, bans placeholder patterns, and handles token-limit splits cleanly. Apply to any task requiring exhaustive, unabridged output.
executing-plans
obra/superpowers
Use when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available