
custom-metrics
Create, track, retrieve, update, and delete custom business metrics for configs. Covers full lifecycle: define metric kinds via API, emit events via SDK, and query results.
- Installs
- 1.8K
- Last 24h
- —
- Trending rank
- —
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/launchdarkly/ai-tooling --skill custom-metrics- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads custom-metrics automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Custom Metrics for Configs
Full lifecycle management of custom business metrics: create metric definitions via API, track events via SDK, retrieve metric data, and manage metrics programmatically.
Prerequisites
- LaunchDarkly SDK initialized (see
sdk) - LaunchDarkly API token with
writerrole for metric management - Understanding of built-in agent metrics (see
built-in-metrics)
API Access
Prefer the LaunchDarkly MCP server tools: they authenticate via OAuth and need no token. If a step has no MCP equivalent and you must call the REST API directly, ask the user to paste an API access token for this session and substitute it for {api_token} in the examples below. Do not search environment variables, .env files, or agent/MCP config files for credentials.
Metrics Lifecycle Overview
| Step | Method | Purpose |
|---|---|---|
| 1. Create | API | Define metric in LaunchDarkly |
| 2. Track | SDK | Send events to the metric |
| 3. Get | API | Retrieve metric definition/data |
| 4. Update | API | Modify metric properties |
| 5. Delete | API | Remove metric |
1. Create Metric (API)
Required fields for numeric custom metrics:
successCriteria- Must be one of:"HigherThanBaseline","LowerThanBaseline"unit- e.g.,"count","percent","milliseconds"
The API will return 400 Bad Request if these are missing for numeric metrics.
import requests
import os
def create_metric(
project_key: str,
metric_key: str,
name: str,
kind: str = "custom",
is_numeric: bool = True,
unit: str = "count",
success_criteria: str = "HigherThanBaseline",
event_key: str = None,
description: str = None
):
"""Create a new metric definition in LaunchDarkly."""
API_TOKEN = "{api_token}" # token the user provided for this session
url = f"https://app.launchdarkly.com/api/v2/metrics/{project_key}"
payload = {
"key": metric_key,
"name": name,
"kind": kind,
"isNumeric": is_numeric,
"eventKey": event_key or metric_key
}
# Unit and successCriteria are required for numeric custom metrics
if is_numeric and kind == "custom":
payload["unit"] = unit
payload["successCriteria"] = success_criteria
if description:
payload["description"] = description
headers = {
"Authorization": API_TOKEN,
"Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
if response.status_code == 201:
print(f"[OK] Created metric: {metric_key}")
return response.json()
elif response.status_code == 409:
print(f"[INFO] Metric already exists: {metric_key}")
return None
else:
print(f"[ERROR] Failed to create metric: {response.status_code}")
print(f" {response.text}")
return None
Metric Kinds:
custom- Track any event (most common for agent metrics)pageview- Track page viewsclick- Track click events
Success Criteria (for numeric metrics):
HigherThanBaseline- Higher values are better (e.g., revenue, satisfaction)LowerThanBaseline- Lower values are better (e.g., errors, latency)
Common Units:
count- Generic countmilliseconds- Time durationpercent- Percentage valuesdollars- Currency
2. Track Events (SDK)
Once the metric is created, track events using the SDK:
from ldclient import Context
from ldclient.config import Config
import ldclient
# Initialize (see sdk for details)
ldclient.set_config(Config("your-sdk-key"))
ld_client = ldclient.get()
def track_metric(ld_client, user_id: str, metric_key: str, value: float, data: dict = None):
"""Track an event to a metric."""
context = Context.builder(user_id).build()
ld_client.track(
metric_key,
context,
data=data,
metric_value=value
)
Common Tracking Patterns
def track_conversion(ld_client, user_id: str, amount: float, config_key: str):
"""Track a conversion event with revenue."""
context = Context.builder(user_id).build()
ld_client.track(
"business.conversion",
context,
data={"configKey": config_key, "category": "electronics"},
metric_value=amount
)
def track_task_success(ld_client, user_id: str, task_type: str, success: bool):
"""Track task completion success/failure."""
context = Context.builder(user_id).build()
ld_client.track(
"task.success_rate",
context,
data={"taskType": task_type},
metric_value=1.0 if success else 0.0
)
def track_satisfaction(ld_client, user_id: str, score: float, feedback_type: str):
"""Track user satisfaction (0-100 scale)."""
context = Context.builder(user_id).build()
ld_client.track(
"user.satisfaction",
context,
data={"feedbackType": feedback_type},
metric_value=score
)
# Track negative feedback separately for alerts
if score < 50:
ld_client.track(
"user.negative_feedback",
context,
metric_value=1.0
)
def track_revenue(ld_client, user_id: str, revenue: float, source: str):
"""Track revenue generated after agent interaction."""
context = Context.builder(user_id).set("tier", "premium").build()
if revenue > 0:
ld_client.track(
"revenue.impact",
context,
data={"source": source},
metric_value=revenue
)
3. Get Metrics (API)
Get Single Metric
def get_metric(project_key: str, metric_key: str):
"""Get a single metric definition."""
API_TOKEN = "{api_token}" # token the user provided for this session
url = f"https://app.launchdarkly.com/api/v2/metrics/{project_key}/{metric_key}"
headers = {"Authorization": API_TOKEN}
response = requests.get(url, headers=headers)
if response.status_code == 200:
metric = response.json()
print(f"[OK] Metric: {metric['key']}")
print(f" Name: {metric.get('name', 'N/A')}")
print(f" Kind: {metric.get('kind', 'N/A')}")
print(f" Numeric: {metric.get('isNumeric', False)}")
print(f" Event Key: {metric.get('eventKey', 'N/A')}")
return metric
elif response.status_code == 404:
print(f"[INFO] Metric not found: {metric_key}")
return None
else:
print(f"[ERROR] Failed to get metric: {response.status_code}")
return None
List All Metrics
def list_metrics(project_key: str, limit: int = 20):
"""List all metrics in a project."""
API_TOKEN = "{api_token}" # token the user provided for this session
url = f"https://app.launchdarkly.com/api/v2/metrics/{project_key}"
headers = {"Authorization": API_TOKEN}
params = {"limit": limit}
response = requests.get(url, headers=headers, params=params)
if response.status_code == 200:
data = response.json()
metrics = data.get("items", [])
print(f"[OK] Found {len(metrics)} metrics:")
for metric in metrics:
numeric = "numeric" if metric.get("isNumeric") else "non-numeric"
print(f" - {metric['key']} ({metric.get('kind', 'custom')}, {numeric})")
return metrics
else:
print(f"[ERROR] Failed to list metrics: {response.status_code}")
return None
4. Update Metric (API)
def update_metric(project_key: str, metric_key: str, updates: list):
"""
Update a metric using JSON Patch operations.
Args:
updates: List of patch operations, e.g.:
[{"op": "replace", "path": "/name", "value": "New Name"}]
"""
API_TOKEN = "{api_token}" # token the user provided for this session
url = f"https://app.launchdarkly.com/api/v2/metrics/{project_key}/{metric_key}"
headers = {
"Authorization": API_TOKEN,
"Content-Type": "application/json"
}
response = requests.patch(url, json=updates, headers=headers)
if response.status_code == 200:
print(f"[OK] Updated metric: {metric_key}")
return response.json()
elif response.status_code == 404:
print(f"[ERROR] Metric not found: {metric_key}")
return None
else:
print(f"[ERROR] Failed to update metric: {response.status_code}")
print(f" {response.text}")
return None
# Example: Update metric name and description
def rename_metric(project_key: str, metric_key: str, new_name: str, new_description: str = None):
"""Rename a metric and optionally update description."""
updates = [
{"op": "replace", "path": "/name", "value": new_name}
]
if new_description:
updates.append({"op": "replace", "path": "/description", "value": new_description})
return update_metric(project_key, metric_key, updates)
5. Delete Metric (API)
def delete_metric(project_key: str, metric_key: str):
"""Delete a metric from the project."""
API_TOKEN = "{api_token}" # token the user provided for this session
url = f"https://app.launchdarkly.com/api/v2/metrics/{project_key}/{metric_key}"
headers = {"Authorization": API_TOKEN}
response = requests.delete(url, headers=headers)
if response.status_code == 204:
print(f"[OK] Deleted metric: {metric_key}")
return True
elif response.status_code == 404:
print(f"[INFO] Metric not found: {metric_key}")
return False
else:
print(f"[ERROR] Failed to delete metric: {response.status_code}")
return False
Complete Workflow Example
import os
import requests
from ldclient import Context
from ldclient.config import Config
import ldclient
# Setup
API_TOKEN = "{api_token}" # token the user provided for this session
SDK_KEY = os.environ.get("LAUNCHDARKLY_SDK_KEY")
PROJECT_KEY = "support-ai"
ldclient.set_config(Config(SDK_KEY))
ld_client = ldclient.get()
# 1. Create metric
create_metric(
PROJECT_KEY,
"ai.task.completion",
name="Agent Task Completion Rate",
kind="custom",
is_numeric=True,
description="Tracks successful agent task completions"
)
# 2. Track events
context = Context.builder("user-123").build()
ld_client.track("ai.task.completion", context, metric_value=1.0)
ld_client.track("ai.task.completion", context, metric_value=1.0)
ld_client.track("ai.task.completion", context, metric_value=0.0) # failure
ld_client.flush()
# 3. Get metric definition
metric = get_metric(PROJECT_KEY, "ai.task.completion")
# 4. Update metric name
rename_metric(PROJECT_KEY, "ai.task.completion", "Agent Task Success Rate")
# 5. List all metrics
list_metrics(PROJECT_KEY)
# 6. Delete metric (when no longer needed)
# delete_metric(PROJECT_KEY, "ai.task.completion")
Session Metrics Tracker
import time
from ldclient import Context
class SessionMetricsTracker:
"""Track metrics across an entire user session."""
def __init__(self, ld_client):
self.ld_client = ld_client
self.session_data = {}
def start_session(self, user_id: str, session_id: str):
"""Initialize session tracking."""
self.session_data[session_id] = {
"user_id": user_id,
"start_time": time.time(),
"interactions": 0,
"successful_tasks": 0
}
def track_interaction(self, session_id: str, success: bool):
"""Track individual interaction within session."""
if session_id not in self.session_data:
return
session = self.session_data[session_id]
session["interactions"] += 1
if success:
session["successful_tasks"] += 1
def end_session(self, session_id: str):
"""Finalize and track session metrics."""
if session_id not in self.session_data:
return None
session = self.session_data[session_id]
duration = time.time() - session["start_time"]
context = Context.builder(session["user_id"]).build()
# Track session duration
self.ld_client.track(
"session.duration",
context,
data={"interactions": session["interactions"]},
metric_value=duration
)
# Track session success rate
if session["interactions"] > 0:
success_rate = session["successful_tasks"] / session["interactions"]
self.ld_client.track(
"session.success_rate",
context,
metric_value=success_rate * 100
)
result = dict(session)
result["duration"] = duration
del self.session_data[session_id]
return result
Naming Conventions
# Use dot notation for hierarchy
"quality.accuracy"
"quality.relevance"
"user.satisfaction"
"user.engagement"
"revenue.conversion"
"task.success_rate"
"session.duration"
"ai.task.completion"
"ai.recommendation.conversion"
Best Practices
- Create Before Track - Metric must exist before tracking events
- Use Numeric Metrics - Set
isNumeric=Truefor aggregation - Consistent Keys - Use same key in
create_metric()andld_client.track() - Always flush before close - Call
ld_client.flush()(await in Node) beforeclose(). Trailing events are at risk of being lost otherwise, in short-lived scripts and long-running services alike. This is not a serverless-only rule; it applies to any process that exits. - Rate Limit - Don't track on every keystroke
Viewing Metrics
Custom metrics appear in:
- Metrics page in LaunchDarkly UI
- Monitoring tab of your config
- Via API using
get_metric()orlist_metrics()
Related Skills
sdk- SDK setupbuilt-in-metrics- Built-in agent metrics (tokens, duration, cost)online-evals- Quality metrics via judges
References
FAQ
What does the custom-metrics skill do?
Create, track, retrieve, update, and delete custom business metrics for configs. Covers full lifecycle: define metric kinds via API, emit events via SDK, and query results.
How do I install custom-metrics?
Run npx skills add https://github.com/launchdarkly/ai-tooling --skill custom-metrics in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/custom-metrics/ (personal) or .claude/skills/custom-metrics/ (project).
Which AI agents can use custom-metrics?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is custom-metrics safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains custom-metrics?
It is published by launchdarkly in launchdarkly/ai-tooling. It has 1,810 recorded installs.
More from launchdarkly/ai-tooling
onboarding
launchdarkly/ai-tooling
Scripted onboarding for LaunchDarkly: quiet execution, fixed sequence, SDK install, first flag with a live reveal, MCP offered afterwards. Enforces step completion before advancing and redirects drift. Use when adding LaunchDarkly, setting up or integrating feature flags in a project, SDK integration, or 'onboard me'.
launchdarkly-flag-discovery
launchdarkly/ai-tooling
Audit your LaunchDarkly feature flags to understand the landscape, find stale or launched flags, and assess removal readiness. Use when the user asks about flag debt, stale flags, cleanup candidates, flag health, or wants to understand their flag inventory.
launchdarkly-flag-create
launchdarkly/ai-tooling
Create and configure LaunchDarkly feature flags in a way that fits the existing codebase. Use when the user wants to create a new flag, wrap code in a flag, add a feature toggle, or set up an experiment. Guides exploration of existing patterns before creating.
launchdarkly-flag-targeting
launchdarkly/ai-tooling
Control LaunchDarkly feature flag targeting including toggling flags on/off, percentage rollouts, targeting rules, individual targets, and copying flag configurations between environments. Use when the user wants to change who sees a flag, roll out to a percentage, add targeting rules, or promote config between environments.
launchdarkly-flag-cleanup
launchdarkly/ai-tooling
Safely remove a feature flag from code while preserving production behavior. Use when the user wants to remove a flag from code, delete flag references, or create a PR that hardcodes the winning variation after a rollout is complete.
launchdarkly-metric-choose
launchdarkly/ai-tooling
Choose the right metrics for a LaunchDarkly experiment, guarded rollout, or release policy. Use when the user wants to know which metrics to use, which is the primary metric for an experiment, what guardrails to add, or which events to monitor in a rollout. Surfaces what will auto-attach from existing release policies before making additional recommendations.
Popular backend & apis skills
find-skills
vercel-labs/skills
Helps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skill should be used when the user is looking for functionality that might exist as an installable skill.
entra-app-registration
microsoft/azure-skills
Guides Microsoft Entra ID app registration, OAuth 2.0 authentication, and MSAL integration. USE FOR: create app registration, register Azure AD app, configure OAuth, set up authentication, add API permissions, generate service principal, MSAL example, console app auth, Entra ID setup, Azure AD authentication. DO NOT USE FOR: Key Vault secrets (use azure-keyvault-expiration-audit), general Azure resource security guidance.
google-agents-cli-adk-code
google/agents-cli
This skill should be used when the user wants to "write agent code", "build an agent with ADK", "add a tool", "create a callback", "define an agent", "use state management" — in a project that needs ADK (Agent Development Kit) API patterns and code examples. It provides a quick reference for agent types, tool definitions, orchestration patterns, callbacks, state management, the graph Workflow API, and reference recipes to study. Do NOT use for scaffolding (use google-agents-cli-scaffold) or deployment (use google-agents-cli-deploy).
embedded-captions
heygen-com/hyperframes
Add captions or subtitles to an existing single-subject talking-head video without editing the footage. Use for plain verbatim captions, cinematic captions embedded behind the subject, VFX captions, “炸/特效/酷炫字幕,” or a named identity from the 35-style catalog. Route by visual identity, not by backend engine. The quiet anchor rail is the default; embed every word only when the user explicitly wants a fully cinematic treatment. The workflow runs locally end to end, including transcription and subject matting; split multi-shot footage before applying it.
entra-agent-id
microsoft/azure-skills
Provision Microsoft Entra Agent Identity Blueprints, BlueprintPrincipals, and per-instance Agent Identities via Microsoft Graph, and configure OAuth 2.0 token exchange (fmipath, OBO, cross-tenant) including the Microsoft Entra SDK for AgentID sidecar. USE FOR: Agent Identity Blueprint, BlueprintPrincipal, agent OAuth, fmipath token exchange, agent OBO, Workload Identity Federation for agents, polyglot agent auth, Microsoft.Identity.Web.AgentIdentities. DO NOT USE FOR: standard Entra app registration (use entra-app-registration), Microsoft Foundry agent authoring (use microsoft-foundry).
neon
neondatabase/agent-skills
Overview of Neon, a complete set of cloud backend primitives around Lakebase Postgres: Auth, Object Storage, Functions, the AI Gateway, and Realtime. Start here to choose Neon for undecided login, files, APIs, and LLM calls, set up the CLI or MCP server, and follow the branch-first workflow. Use when building an app or backend on Neon, or when "Neon" or "Lakebase Postgres" is mentioned. Child skill neon-postgres wins for an existing DATABASEURL, SQL, schema, inspect, or search. Child skill neon-auth wins for login, users, sessions, identity routing, and Managed Better Auth setup. Child skill neon-realtime wins for live queries in reactive apps and agents. Also use for object storage, S3, buckets, serverless functions, function triggers, cron, AI gateway, LLM calls, realtime, logs, Loki, Grafana, observability, postgres, database, backend, Claimable Neon, neon.new, or a no-signup database.