arize-ai-provider-integration
Creates, reads, updates, and deletes Arize AI integrations that store LLM provider credentials used by evaluators and other Arize features. Supports any LLM provider (e.g. OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, NVIDIA NIM). Use when the user mentions AI integration, LLM provider credentials, create integration, list integrations, update credentials, delete integration, or connecting an LLM provider to Arize.
- Installs
- 1.1K
- Last 24h
- —
- Trending rank
- —
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/github/awesome-copilot --skill arize-ai-provider-integration- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads arize-ai-provider-integration automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Arize AI Integration Skill
SPACE— Most--spaceflags and theARIZE_SPACEenv var accept a space name (e.g.,my-workspace) or a base64 space ID (e.g.,U3BhY2U6...). Find yours withax spaces list. Note:ai-integrations createdoes not accept--space— AI integrations are account-scoped. Use--spaceonly withlist,get,update, anddelete.
Concepts
- AI Integration = stored LLM provider credentials registered in Arize; used by evaluators to call a judge model and by other Arize features that need to invoke an LLM on your behalf
- Provider = the LLM service backing the integration (e.g.,
openAI,anthropic,awsBedrock) - Integration ID = a base64-encoded global identifier for an integration (e.g.,
TGxtSW50ZWdyYXRpb246MTI6YUJjRA==); required for evaluator creation and other downstream operations - Scoping = visibility rules controlling which spaces or users can use an integration
- Auth type = how Arize authenticates with the provider:
default(provider API key),proxy_with_headers(proxy via custom headers), orbearer_token(bearer token auth)
Prerequisites
Proceed directly with the task — run the ax command you need. Do NOT check versions, env vars, or profiles upfront.
If an ax command fails, troubleshoot based on the error:
command not foundor version error → see references/ax-setup.md401 Unauthorized/ missing API key → runax profiles showto inspect the current profile. If the profile is missing or the API key is wrong, follow references/ax-profiles.md to create/update it. If the user doesn't have their key, direct them to https://app.arize.com/admin > API Keys- Space unknown → run
ax spaces listto pick by name, or ask the user - LLM provider call fails (missing OPENAI_API_KEY / ANTHROPIC_API_KEY) → run
ax ai-integrations list --space SPACEto check for platform-managed credentials. If none exist, ask the user to provide the key or create an integration via the arize-ai-provider-integration skill - Security: Never read
.envfiles or search the filesystem for credentials. Useax profilesfor Arize credentials andax ai-integrationsfor LLM provider keys. If credentials are not available through these channels, ask the user.
List AI Integrations
List all integrations accessible in a space:
ax ai-integrations list --space SPACE
Filter by name (case-insensitive substring match):
ax ai-integrations list --space SPACE --name "openai"
Paginate large result sets:
# Get first page
ax ai-integrations list --space SPACE --limit 20 -o json
# Get next page using cursor from previous response
ax ai-integrations list --space SPACE --limit 20 --cursor CURSOR_TOKEN -o json
Key flags:
| Flag | Description |
|---|---|
--space | Space name or ID to filter integrations |
--name | Case-insensitive substring filter on integration name |
--limit | Max results (1–100, default 15) |
--cursor | Pagination token from a previous response |
-o, --output | Output format: table (default) or json |
Response fields:
| Field | Description |
|---|---|
id | Base64 integration ID — copy this for downstream commands |
name | Human-readable name |
provider | LLM provider enum (see Supported Providers below) |
has_api_key | true if credentials are stored |
model_names | Allowed model list, or null if all models are enabled |
enable_default_models | Whether default models for this provider are allowed |
function_calling_enabled | Whether tool/function calling is enabled |
auth_type | Authentication method: default, proxy_with_headers, or bearer_token |
Get a Specific Integration
ax ai-integrations get NAME_OR_ID
ax ai-integrations get NAME_OR_ID -o json
ax ai-integrations get NAME_OR_ID --space SPACE # required when using name instead of ID
Use this to inspect an integration's full configuration or to confirm its ID after creation.
Create an AI Integration
Before creating, always list integrations first — the user may already have a suitable one:
ax ai-integrations list --space SPACE
If no suitable integration exists, create one. The required flags depend on the provider.
OpenAI
ax ai-integrations create \
--name "My OpenAI Integration" \
--provider openAI \
--api-key $OPENAI_API_KEY
Anthropic
ax ai-integrations create \
--name "My Anthropic Integration" \
--provider anthropic \
--api-key $ANTHROPIC_API_KEY
Azure OpenAI
ax ai-integrations create \
--name "My Azure OpenAI Integration" \
--provider azureOpenAI \
--api-key $AZURE_OPENAI_API_KEY \
--base-url "https://my-resource.openai.azure.com/"
AWS Bedrock
AWS Bedrock uses IAM role-based auth. Provide the ARN of the role Arize should assume via --provider-metadata:
ax ai-integrations create \
--name "My Bedrock Integration" \
--provider awsBedrock \
--provider-metadata '{"role_arn": "arn:aws:iam::123456789012:role/ArizeBedrockRole"}'
Vertex AI
Vertex AI uses GCP service account credentials. Provide the GCP project and region via --provider-metadata:
ax ai-integrations create \
--name "My Vertex AI Integration" \
--provider vertexAI \
--provider-metadata '{"project_id": "my-gcp-project", "location": "us-central1"}'
Gemini
ax ai-integrations create \
--name "My Gemini Integration" \
--provider gemini \
--api-key $GEMINI_API_KEY
NVIDIA NIM
ax ai-integrations create \
--name "My NVIDIA NIM Integration" \
--provider nvidiaNim \
--api-key $NVIDIA_API_KEY \
--base-url "https://integrate.api.nvidia.com/v1"
Custom (OpenAI-compatible endpoint)
ax ai-integrations create \
--name "My Custom Integration" \
--provider custom \
--base-url "https://my-llm-proxy.example.com/v1" \
--api-key $CUSTOM_LLM_API_KEY
Supported Providers
| Provider | Required extra flags |
|---|---|
openAI | --api-key <key> |
anthropic | --api-key <key> |
azureOpenAI | --api-key <key>, --base-url <azure-endpoint> |
awsBedrock | --provider-metadata '{"role_arn": "<arn>"}' |
vertexAI | --provider-metadata '{"project_id": "<gcp-project>", "location": "<region>"}' |
gemini | --api-key <key> |
nvidiaNim | --api-key <key>, --base-url <nim-endpoint> |
custom | --base-url <endpoint> |
Optional flags for any provider
| Flag | Description |
|---|---|
--model-name | Allowed model name (repeat for multiple, e.g. --model-name gpt-4o --model-name gpt-4o-mini); omit to allow all models |
--enable-default-models | Enable the provider's default model list |
--function-calling-enabled | Enable tool/function calling support |
--auth-type | Authentication type: default, proxy_with_headers, or bearer_token |
--headers | Custom headers as JSON object or file path (for proxy auth) |
--provider-metadata | Provider-specific metadata as JSON object or file path |
After creation
Capture the returned integration ID (e.g., TGxtSW50ZWdyYXRpb246MTI6YUJjRA==) — it is needed for evaluator creation and other downstream commands. If you missed it, retrieve it:
ax ai-integrations list --space SPACE -o json
# or by name/ID directly:
ax ai-integrations get NAME_OR_ID
Update an AI Integration
update is a partial update — only the flags you provide are changed. Omitted fields stay as-is.
# Rename
ax ai-integrations update NAME_OR_ID --name "New Name"
# Rotate the API key
ax ai-integrations update NAME_OR_ID --api-key $OPENAI_API_KEY
# Change the model list (replaces all existing model names)
ax ai-integrations update NAME_OR_ID --model-name gpt-4o --model-name gpt-4o-mini
# Update base URL (for Azure, custom, or NIM)
ax ai-integrations update NAME_OR_ID --base-url "https://new-endpoint.example.com/v1"
Add --space SPACE when using a name instead of ID. Any flag accepted by create can be passed to update.
Delete an AI Integration
Warning: Deletion is permanent. Evaluators that reference this integration will no longer be able to run.
ax ai-integrations delete NAME_OR_ID --force
ax ai-integrations delete NAME_OR_ID --space SPACE --force # required when using name instead of ID
Omit --force to get a confirmation prompt instead of deleting immediately.
Troubleshooting
| Problem | Solution |
|---|---|
ax: command not found | See references/ax-setup.md |
401 Unauthorized | API key may not have access to this space. Verify key and space ID at https://app.arize.com/admin > API Keys |
No profile found | Run ax profiles show --expand; set ARIZE_API_KEY env var or write ~/.arize/config.toml |
Integration not found | Verify with ax ai-integrations list --space SPACE |
has_api_key: false after create | Credentials were not saved — re-run update with the correct --api-key or --provider-metadata |
| Evaluator runs fail with LLM errors | Check integration credentials with ax ai-integrations get INT_ID; rotate the API key if needed |
provider mismatch | Cannot change provider after creation — delete and recreate with the correct provider |
Related Skills
- arize-evaluator: Create LLM-as-judge evaluators that use an AI integration → use
arize-evaluator - arize-experiment: Run experiments that use evaluators backed by an AI integration → use
arize-experiment
Save Credentials for Future Use
See references/ax-profiles.md § Save Credentials for Future Use.
FAQ
What does the arize-ai-provider-integration skill do?
Creates, reads, updates, and deletes Arize AI integrations that store LLM provider credentials used by evaluators and other Arize features. Supports any LLM provider (e.g. OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, NVIDIA NIM). Use when the user mentions AI integration, LLM provider credentials, create integration, list integrations, update credentials, delete integration, or connecting an LLM provider to Arize.
How do I install arize-ai-provider-integration?
Run npx skills add https://github.com/github/awesome-copilot --skill arize-ai-provider-integration in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/arize-ai-provider-integration/ (personal) or .claude/skills/arize-ai-provider-integration/ (project).
Which AI agents can use arize-ai-provider-integration?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is arize-ai-provider-integration safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains arize-ai-provider-integration?
It is published by github in github/awesome-copilot, and it is an official skill from the maker of the technology it covers. It has 1,097 recorded installs.
More from github/awesome-copilot
git-commit
github/awesome-copilot
Use this skill when the user wants to make a Git commit or split changes into more than one commit. Also use it when the user types "/commit" or asks for a conventional commit. The skill examines the diff, stages the related files, writes a Conventional Commits message, and runs git commit. Do not use it when the user wants only a commit message and no commit. Do not use it to push, open a pull request, rebase, or change the commit history.
excalidraw-diagram-generator
github/awesome-copilot
Generate Excalidraw diagrams from natural language descriptions. Use when asked to "create a diagram", "make a flowchart", "visualize a process", "draw a system architecture", "create a mind map", or "generate an Excalidraw file". Supports flowcharts, relationship diagrams, mind maps, and system architecture diagrams. Outputs .excalidraw JSON files that can be opened directly in Excalidraw.
multi-stage-dockerfile
github/awesome-copilot
Create optimized multi-stage Dockerfiles for any language or framework
documentation-writer
github/awesome-copilot
Diátaxis Documentation Expert. An expert technical writer specializing in creating high-quality software documentation, guided by the principles and structure of the Diátaxis technical documentation authoring framework.
prd
github/awesome-copilot
Generate high-quality Product Requirements Documents (PRDs) for software systems and AI-powered features. Includes executive summaries, user stories, technical specifications, and risk analysis.
refactor
github/awesome-copilot
Surgical code refactoring to improve maintainability without changing behavior. Covers extracting functions, renaming variables, breaking down god functions, improving type safety, eliminating code smells, and applying design patterns. Less drastic than repo-rebuilder; use for gradual improvements.
Popular ai & agents skills
google-agents-cli-publish
google/agents-cli
This skill should be used when the user wants to "publish an agent", "publish my ADK agent", "register an agent with Gemini", "publish to Gemini" (formerly Gemini Enterprise), or needs guidance on the agents-cli publish gemini command. Also use when the user wants to "manage agents in Agent Registry", "list/update/delete registered agents", or "register an MCP server". Covers ADK vs A2A registration modes, programmatic and interactive usage, flag reference, auto-detection from deployment metadata, Agent Registry fleet management, and troubleshooting. Part of the agents-cli skills suite. Do NOT use for deployment (use google-agents-cli-deploy).
google-agents-cli-eval
google/agents-cli
This skill should be used when the user wants to "run an evaluation", "evaluate my agent", "evaluate my ADK agent", "write an eval dataset", "analyze eval failures", "compare eval results", "optimize agent", or needs guidance on the Gemini platform eval methodology and the Quality Flywheel. Covers eval metrics, dataset schema, LLM-as-judge scoring, and common failure causes. Applies to any agents-cli project, whatever framework the agent is written in. Do NOT use for agent API code patterns (ADK: use google-agents-cli-adk-code), deployment (use google-agents-cli-deploy), or project scaffolding (use google-agents-cli-scaffold).
skill-creator
anthropics/skills
Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.
codex-pet
prime-skills/runcomfy-agent-skills
Codex Pet generator on RunComfy. Build a Codex-compatible Codex Pet spritesheet.webp + pet.json from a single reference image, drop it into ${CODEXHOME:-$HOME/.codex}/pets/ / and Codex picks it up as a custom Codex Pet next to the 8 built-ins. This skill produces the exact Codex Pet atlas Codex expects (1536x1872 PNG/WebP, 8 cols x 9 rows, 192x208 cells, 9 animation states — idle, running-right, running-left, waving, jumping, failed, waiting, running, review). Calls OpenAI GPT Image 2 edit ONCE via the local RunComfy CLI as runcomfy run openai/gpt-image-2/edit to produce a canonical Codex Pet pose, then assembles all 9 animation rows programmatically with ImageMagick micro-transforms — no Codex Pro, no $imagegen, no OPENAIAPIKEY required, only RUNCOMFYTOKEN. Triggers on "codex pet", "create codex pet", "make codex pet", "hatch codex pet", "/hatch image", "desktop pet codex", "codex pets", "spritesheet.webp", or any explicit ask to build a custom pet for OpenAI Codex.
full-output-enforcement
leonxlnx/taste-skill
Overrides default LLM truncation behavior. Enforces complete code generation, bans placeholder patterns, and handles token-limit splits cleanly. Apply to any task requiring exhaustive, unabridged output.
executing-plans
obra/superpowers
Use when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available