MCPInstall

deepeval-tracing

by confident-aiAI & Agents

Instrument an AI application with DeepEval's native tracing so its behavior is visible in Confident AI. TRIGGER when the user wants to add DeepEval tracing or @observe to an LLM app, agent, RAG pipeline, or chatbot; wire a framework, model-provider, or vector-database integration (LangGraph, LangChain, OpenAI Agents, LlamaIndex, Pydantic AI, CrewAI, and others); choose between a native integration and manual instrumentation; set span types, tags, or metadata; or send DeepEval-SDK traces to Confident AI's Observatory. DO NOT TRIGGER for building DeepEval pytest eval suites, datasets, goldens, metrics, or deepeval test run (use the deepeval skill), or for raw OpenTelemetry / OTLP export without the deepeval package (use the deepeval-otel skill). This skill is purely DeepEval-SDK instrumentation — producing well-formed traces, not running evals.

Installs
82
Last 24h
—
Trending rank
—
Security
Not audited

Install

Works with every agent the skills CLI supports.

npx skills add https://github.com/confident-ai/deepeval --skill deepeval-tracing
  1. 01

    Install

    Run the command in your project. The CLI asks which agents to install for.

  2. 02

    Describe the task

    Ask your agent for something this skill covers. It loads deepeval-tracing automatically when relevant.

  3. 03

    Review the result

    The agent follows the SKILL.md steps, scripts and references bundled with the skill.

SKILL.md

The instructions your agent reads when the skill is triggered.

DeepEval Tracing

Use this skill to instrument an AI application — an LLM app, agent, RAG pipeline, or chatbot — with DeepEval's native tracing so its execution is visible span by span in Confident AI's Observatory. The work is: pick a supported integration when one exists, fall back to manual @observe otherwise, give each span a meaningful type, and add tags and metadata.

This skill stops at producing well-formed traces. Attaching evaluation metrics and running evals is the deepeval skill's job.

Scope: AI Applications Only

Instrument only the AI parts of the system — agent loops and planning, LLM calls, retrieval / vector search, and tool calls. The span types (llm, retriever, tool, agent) describe AI components. Do not trace non-AI software (web servers, CRUD backends, infrastructure). If the target has no LLM, agent, retrieval, or tool-calling component, this skill does not apply.

When to Use vs the deepeval and deepeval-otel Skills

  • This skill (deepeval-tracing) — instrument an app with the DeepEval SDK (@observe, framework integrations) so traces reach Confident AI.
  • deepeval skill — build pytest eval suites: datasets, metrics, traced evals, deepeval test run, iteration. It runs evals against an app this skill instrumented.
  • deepeval-otel skill — instrument with the vendor-neutral OpenTelemetry SDK instead of the DeepEval SDK (raw OTLP, including non-Python apps).

The three are complementary. If unsure between this skill and deepeval-otel: use this one when the app is Python and you want the DeepEval SDK; use deepeval-otel when you want raw OpenTelemetry or the app is not Python.

Prerequisites

  • An AI application in Python with pip install deepeval.
  • For traces to reach Confident AI: deepeval login, or an exported CONFIDENT_API_KEY (preferred for CI and non-interactive runs).

Workflow

  1. Confirm the target is an AI application (it has LLM calls, an agent loop, retrieval, or tool calls). If it has none of these, stop — this skill does not apply.
  2. Detect the framework, model provider, agent SDK, and vector database in use.
  3. Read references/integrations.md and the exact integration doc for what was detected. Prefer a native integration over manual instrumentation.
  4. If no native integration fits, instrument manually with @observe. Read references/tracing.md.
  5. Give each span a meaningful type (llm, retriever, tool, agent) and capture inputs/outputs.
  6. Add trace-level tags and metadata where they help diagnose failure patterns. Never trace secrets, credentials, or raw sensitive data.
  7. Confirm deepeval login or CONFIDENT_API_KEY, then verify traces appear in the Confident AI Observatory.

Core Principles

  1. Instrument AI components only — llm, retriever, tool, agent spans. Never trace non-AI software.
  2. Prefer a supported integration over manual @observe. Manual tracing is the fallback for unsupported frameworks and app-owned wrapper boundaries.
  3. Read the exact integration doc before writing tracing code.
  4. Give spans meaningful types; let names default to function names unless there is a strong reason to override.
  5. Never trace secrets, credentials, API keys, or raw sensitive user data.
  6. Producing traces is the scope. Attaching metrics and running evals belong to the deepeval skill; raw OpenTelemetry export belongs to deepeval-otel.

References

TopicFile
Manual instrumentation: @observe, span types, tags, metadatareferences/tracing.md
Integration selection rule and framework / model / vector-DB doc indexreferences/integrations.md

FAQ

What does the deepeval-tracing skill do?

Instrument an AI application with DeepEval's native tracing so its behavior is visible in Confident AI. TRIGGER when the user wants to add DeepEval tracing or @observe to an LLM app, agent, RAG pipeline, or chatbot; wire a framework, model-provider, or vector-database integration (LangGraph, LangChain, OpenAI Agents, LlamaIndex, Pydantic AI, CrewAI, and others); choose between a native integration and manual instrumentation; set span types, tags, or metadata; or send DeepEval-SDK traces to Confident AI's Observatory. DO NOT TRIGGER for building DeepEval pytest eval suites, datasets, goldens, metrics, or deepeval test run (use the deepeval skill), or for raw OpenTelemetry / OTLP export without the deepeval package (use the deepeval-otel skill). This skill is purely DeepEval-SDK instrumentation — producing well-formed traces, not running evals.

How do I install deepeval-tracing?

Run npx skills add https://github.com/confident-ai/deepeval --skill deepeval-tracing in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/deepeval-tracing/ (personal) or .claude/skills/deepeval-tracing/ (project).

Which AI agents can use deepeval-tracing?

Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.

Is deepeval-tracing safe to install?

No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.

Who maintains deepeval-tracing?

It is published by confident-ai in confident-ai/deepeval. It has 82 recorded installs.

More from confident-ai/deepeval

View all →

deepeval

confident-ai/deepeval

DeepEval evaluation workflow for AI agents and LLM applications. TRIGGER when the user wants to evaluate or improve an AI agent, tool-using workflow, multi-turn chatbot, RAG pipeline, or LLM app; add evals; generate datasets or goldens; use deepeval generate; use deepeval test run; send results to Confident AI; monitor production; run online evals; inspect traces; or iterate on prompts, tools, retrieval, or agent behavior from eval failures. AI agents are the primary use case. Covers Python SDK, pytest eval suites, CLI generation, traced evals, Confident AI reporting, and agent-driven improvement loops. DO NOT TRIGGER for unrelated generic pytest, non-AI test setup, or non-DeepEval observability work unless the user asks to compare or migrate to DeepEval; for instrumenting an app with DeepEval tracing, @observe, or framework integrations (use the deepeval-tracing skill); or for raw OpenTelemetry / OTLP export without the deepeval package (use the deepeval-otel skill).

AI & Agents2.8K installs

deepeval-otel

confident-ai/deepeval

Export raw OpenTelemetry traces from an AI application to Confident AI's Observatory. TRIGGER when the user wants to send OpenTelemetry or OTLP traces/spans from an LLM app, agent, RAG pipeline, or chatbot to Confident AI; configure the Confident AI OTLP endpoint; set confident.span. or confident.trace. attributes; export AI-app traces without the deepeval Python package; wire an OTLPSpanExporter, OpenTelemetry Collector, or vendor-neutral OTel SDK to Confident AI; or pick the US vs EU OTLP endpoint. Language-agnostic: the mechanism is OTLP attribute keys plus an exporter endpoint. DO NOT TRIGGER for building DeepEval pytest eval suites, datasets, goldens, metrics, or deepeval test run (use the deepeval skill); for instrumenting with the DeepEval SDK's @observe decorator or framework integrations (use the deepeval-tracing skill); or for non-AI software such as web servers, CRUD backends, or infrastructure: the confident. attributes describe AI components only.

AI & Agents41 installs

Popular ai & agents skills

View category →

google-agents-cli-publish

google/agents-cli

This skill should be used when the user wants to "publish an agent", "publish my ADK agent", "register an agent with Gemini", "publish to Gemini" (formerly Gemini Enterprise), or needs guidance on the agents-cli publish gemini command. Also use when the user wants to "manage agents in Agent Registry", "list/update/delete registered agents", or "register an MCP server". Covers ADK vs A2A registration modes, programmatic and interactive usage, flag reference, auto-detection from deployment metadata, Agent Registry fleet management, and troubleshooting. Part of the agents-cli skills suite. Do NOT use for deployment (use google-agents-cli-deploy).

AI & Agents513.9K installs

google-agents-cli-eval

google/agents-cli

This skill should be used when the user wants to "run an evaluation", "evaluate my agent", "evaluate my ADK agent", "write an eval dataset", "analyze eval failures", "compare eval results", "optimize agent", or needs guidance on the Gemini platform eval methodology and the Quality Flywheel. Covers eval metrics, dataset schema, LLM-as-judge scoring, and common failure causes. Applies to any agents-cli project, whatever framework the agent is written in. Do NOT use for agent API code patterns (ADK: use google-agents-cli-adk-code), deployment (use google-agents-cli-deploy), or project scaffolding (use google-agents-cli-scaffold).

AI & Agents513.5K installs

skill-creator

anthropics/skills

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

OfficialAI & Agents416.4K installs

codex-pet

prime-skills/runcomfy-agent-skills

Codex Pet generator on RunComfy. Build a Codex-compatible Codex Pet spritesheet.webp + pet.json from a single reference image, drop it into ${CODEXHOME:-$HOME/.codex}/pets/ / and Codex picks it up as a custom Codex Pet next to the 8 built-ins. This skill produces the exact Codex Pet atlas Codex expects (1536x1872 PNG/WebP, 8 cols x 9 rows, 192x208 cells, 9 animation states — idle, running-right, running-left, waving, jumping, failed, waiting, running, review). Calls OpenAI GPT Image 2 edit ONCE via the local RunComfy CLI as runcomfy run openai/gpt-image-2/edit to produce a canonical Codex Pet pose, then assembles all 9 animation rows programmatically with ImageMagick micro-transforms — no Codex Pro, no $imagegen, no OPENAIAPIKEY required, only RUNCOMFYTOKEN. Triggers on "codex pet", "create codex pet", "make codex pet", "hatch codex pet", "/hatch image", "desktop pet codex", "codex pets", "spritesheet.webp", or any explicit ask to build a custom pet for OpenAI Codex.

AI & Agents402.1K installs

full-output-enforcement

leonxlnx/taste-skill

Overrides default LLM truncation behavior. Enforces complete code generation, bans placeholder patterns, and handles token-limit splits cleanly. Apply to any task requiring exhaustive, unabridged output.

AI & Agents401K installs

executing-plans

obra/superpowers

Use when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available

AI & Agents245.7K installs