instrumenting-with-mlflow-tracing
Add or repair MLflow tracing in Python or TypeScript applications, including tracing required for evaluation. Covers framework autologging, custom spans, backend setup, and persisted trace verification.
- Installs
- 1.1K
- Last 24h
- +8
- Trending rank
- #9564
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/mlflow/skills --skill instrumenting-with-mlflow-tracing- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads instrumenting-with-mlflow-tracing automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Instrument MLflow tracing
Reuse working tracing for investigation or evaluation.
Prerequisites
- Check entry point, language/framework, SDK version, tracking URI, experiment, and deployment identity.
- Preserve destinations and providers. Explicit application destinations override inherited environment values; assistant telemetry may use another experiment.
- Read Python or TypeScript for the runtime.
Step 1: Check the destination (Databricks only)
For OSS or SageMaker, keep the configured backend and proceed to Step 2.
- Read UC setup before editing; validate auth/workspace.
- Reuse the explicit destination; otherwise resolve schema choice before provisioning.
- Discover an accessible warehouse automatically. Default to UC; legacy storage requires an explicit request.
Step 2: Enable framework tracing
- Enable the matching integration before execution. LangGraph uses
mlflow.langchain.autolog(), notmlflow.langgraph.autolog(). - Do not wrap automatically traced nodes, tools, or models again. Reserve manual spans for uncaptured work or an outer application boundary.
Step 3: Capture application work
- Trace meaningful root, LLM, retrieval, tool, routing, and external I/O (files/queues). Skip utility, configuration, and logging noise.
- Capture actual relevant inputs/outputs explicitly for manual spans; redact before export.
Step 4: Run one request
- Run one representative request; record start/end epoch milliseconds.
- Save its trace ID immediately in-process:
mlflow.get_last_active_trace_id()in Python; the active span's ID in TypeScript. - Write/flush the trace ID to a file or captured stdout before readback; retain it if the process is cancelled. Do not rerun to recover an ID.
<a id="verification"></a>
Verification
-
Fetch a persisted application trace, not a local span or old trace.
-
Set a 60-second wall-clock process deadline for readback, including flush, SDK retries, requests, and backoff. Terminate at expiry unless the user explicitly requests deeper troubleshooting.
-
Tool yields/thread-wait timeouts do not cancel calls. Without cancellation, report backend verification as unperformed.
-
Flush once (
mlflow.flush_trace_async_logging()/await mlflow.flushTraces()); fetch the saved ID. -
Without an ID, search once in the configured experiment with run-time bounds,
max_results=1, and spans included. -
Every UC search needs
trace.timestamp_ms; result limits do not bound scans. Pythonsearch_traces()has nostart_timekeyword.
For Python readback, adapt this recipe to the saved run values and enforce the deadline externally:
mlflow.flush_trace_async_logging()
if trace_id:
trace = mlflow.get_trace(trace_id)
else:
traces = mlflow.search_traces(
locations=[experiment_id],
filter_string=(f"trace.timestamp_ms >= {run_start_ms} "
f"AND trace.timestamp_ms <= {run_end_ms}"),
max_results=1, return_type="list", include_spans=True,
)
trace = traces[0] if traces else None
assert trace is not None, "No persisted trace read back"
assert run_start_ms <= trace.info.request_time <= run_end_ms
assert trace.data.spans, "No spans captured"
-
Inspect expected root/graph, model, retrieval, and tool spans and actual I/O. Confirm the application's request, not scorer or assistant telemetry.
-
Check experiment/location from returned metadata, or the bound UC location if no experiment ID exists.
-
Reuse returned spans; stop after success. Available
agent-evaluation/scripts/validate_tracing_runtime.pymay replace this check under the same deadline, not add another audit. -
Retry at most once after a concrete quick fix within the original budget.
-
Stop on missing provider keys, auth/permission errors, warehouse timeouts, or
RESOURCE_EXHAUSTED/429. -
No parallel searches, repeated agent runs, direct UC SQL, raw REST, alternate credentials, or warehouse fallbacks.
-
Report operation/error, captured ID, successes, and unverified results. Request the missing prerequisite, not secrets.
-
Keep instrumentation changes if backend verification is blocked; pause evaluation until tracing is verified.
Return the verified trace ID/link and observed spans. For Databricks use the verified trace URL rules, not an experiment-only link. Do not label an unverified link as proof.
References
-
Deployment, async export, sampling: production.
-
Async/thread context, redaction, caught errors or links: advanced patterns.
-
Cross-service parent propagation: distributed tracing.
-
User ratings tied to traces: feedback collection.
-
Unreadable non-chat root previews: root span previews.
-
Explaining captured behavior:
debug-agent; implementing a requested behavior fix:fix-agent-issue, carrying the trace evidence. These are optional follow-on tasks. -
Other integrations: automatic tracing docs. Check installed signatures/types.
-
Missing/moved topic: llms.txt. Docs unavailable: use local recipes and installed API introspection; stop if correctness remains uncertain.
FAQ
What does the instrumenting-with-mlflow-tracing skill do?
Add or repair MLflow tracing in Python or TypeScript applications, including tracing required for evaluation. Covers framework autologging, custom spans, backend setup, and persisted trace verification.
How do I install instrumenting-with-mlflow-tracing?
Run npx skills add https://github.com/mlflow/skills --skill instrumenting-with-mlflow-tracing in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/instrumenting-with-mlflow-tracing/ (personal) or .claude/skills/instrumenting-with-mlflow-tracing/ (project).
Which AI agents can use instrumenting-with-mlflow-tracing?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is instrumenting-with-mlflow-tracing safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains instrumenting-with-mlflow-tracing?
It is published by mlflow in mlflow/skills. It has 1,056 recorded installs.
More from mlflow/skills
mlflow-onboarding
mlflow/skills
Onboards users to MLflow by determining their use case (GenAI agents/apps or traditional ML/deep learning) and guiding them through relevant quickstart tutorials and initial integration. If an experiment ID is available, it should be supplied as input to help determine the use case. Use when the user asks to get started with MLflow, set up tracking, add observability, or integrate MLflow into their project. Triggers on "get started with MLflow", "set up MLflow", "onboard to MLflow", "add MLflow to my project", "how do I use MLflow".
agent-evaluation
mlflow/skills
Use this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality - including improving tool selection accuracy, answer quality, reducing costs, or fixing issues where the agent gives wrong/incomplete responses. Evaluates agents systematically using MLflow evaluation with datasets, scorers, and tracing. IMPORTANT - Always also load the instrumenting-with-mlflow-tracing skill before starting any work. Covers end-to-end evaluation workflow or individual components (tracing setup, dataset creation, scorer definition, evaluation execution).
searching-mlflow-docs
mlflow/skills
Searches and retrieves MLflow documentation from the official docs site. Use when the user asks about MLflow features, APIs, integrations (LangGraph, LangChain, OpenAI, etc.), tracing, tracking, or requests to look up MLflow documentation. Triggers on "how do I use MLflow with X", "find MLflow docs for Y", "MLflow API for Z".
analyzing-mlflow-trace
mlflow/skills
Analyzes a single MLflow trace to answer a user query about it. Use when the user provides a trace ID and asks to debug, investigate, find issues, root-cause errors, understand behavior, or analyze quality. Triggers on "analyze this trace", "what went wrong with this trace", "debug trace", "investigate trace", "why did this trace fail", "root cause this trace".
retrieving-mlflow-traces
mlflow/skills
Retrieves MLflow traces using CLI or Python API. Use when the user asks to get a trace by ID, find traces, filter traces by status/tags/metadata/execution time, query traces, or debug failed traces. Triggers on "get trace", "search traces", "find failed traces", "filter traces by", "traces slower than", "query MLflow traces".
analyzing-mlflow-session
mlflow/skills
Analyzes an MLflow session — a sequence of traces from a multi-turn chat conversation or interaction. Use when the user asks to debug a chat conversation, review session or chat history, find where a multi-turn chat went wrong, or analyze patterns across turns. Triggers on "analyze this session", "what happened in this conversation", "debug session", "review chat history", "where did this chat go wrong", "session traces", "analyze chat", "debug this chat".
Popular backend & apis skills
find-skills
vercel-labs/skills
Helps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skill should be used when the user is looking for functionality that might exist as an installable skill.
entra-app-registration
microsoft/azure-skills
Guides Microsoft Entra ID app registration, OAuth 2.0 authentication, and MSAL integration. USE FOR: create app registration, register Azure AD app, configure OAuth, set up authentication, add API permissions, generate service principal, MSAL example, console app auth, Entra ID setup, Azure AD authentication. DO NOT USE FOR: Key Vault secrets (use azure-keyvault-expiration-audit), general Azure resource security guidance.
google-agents-cli-adk-code
google/agents-cli
This skill should be used when the user wants to "write agent code", "build an agent with ADK", "add a tool", "create a callback", "define an agent", "use state management" — in a project that needs ADK (Agent Development Kit) API patterns and code examples. It provides a quick reference for agent types, tool definitions, orchestration patterns, callbacks, state management, the graph Workflow API, and reference recipes to study. Do NOT use for scaffolding (use google-agents-cli-scaffold) or deployment (use google-agents-cli-deploy).
embedded-captions
heygen-com/hyperframes
Add captions or subtitles to an existing single-subject talking-head video without editing the footage. Use for plain verbatim captions, cinematic captions embedded behind the subject, VFX captions, “炸/特效/酷炫字幕,” or a named identity from the 35-style catalog. Route by visual identity, not by backend engine. The quiet anchor rail is the default; embed every word only when the user explicitly wants a fully cinematic treatment. The workflow runs locally end to end, including transcription and subject matting; split multi-shot footage before applying it.
entra-agent-id
microsoft/azure-skills
Provision Microsoft Entra Agent Identity Blueprints, BlueprintPrincipals, and per-instance Agent Identities via Microsoft Graph, and configure OAuth 2.0 token exchange (fmipath, OBO, cross-tenant) including the Microsoft Entra SDK for AgentID sidecar. USE FOR: Agent Identity Blueprint, BlueprintPrincipal, agent OAuth, fmipath token exchange, agent OBO, Workload Identity Federation for agents, polyglot agent auth, Microsoft.Identity.Web.AgentIdentities. DO NOT USE FOR: standard Entra app registration (use entra-app-registration), Microsoft Foundry agent authoring (use microsoft-foundry).
neon
neondatabase/agent-skills
Overview of Neon, a complete set of cloud backend primitives around Lakebase Postgres: Auth, Object Storage, Functions, the AI Gateway, and Realtime. Start here to choose Neon for undecided login, files, APIs, and LLM calls, set up the CLI or MCP server, and follow the branch-first workflow. Use when building an app or backend on Neon, or when "Neon" or "Lakebase Postgres" is mentioned. Child skill neon-postgres wins for an existing DATABASEURL, SQL, schema, inspect, or search. Child skill neon-auth wins for login, users, sessions, identity routing, and Managed Better Auth setup. Child skill neon-realtime wins for live queries in reactive apps and agents. Also use for object storage, S3, buckets, serverless functions, function triggers, cron, AI gateway, LLM calls, realtime, logs, Loki, Grafana, observability, postgres, database, backend, Claimable Neon, neon.new, or a no-signup database.