querying-aws-sagemaker-catalog
Runs SQL analytics on SageMaker Catalog asset metadata tables exported as Apache Iceberg in S3 Tables. Covers governance queries, asset growth tracking, ownership audits, time-travel over catalog state, and metadata quality analysis. Applies when querying catalog inventory, finding assets without descriptions, comparing catalog snapshots, or auditing data ownership. Trigger phrases: catalog inventory SQL, how many assets, assets without descriptions, asset growth over time, who owns this data, catalog governance, data quality audit, catalog analytics.
- Installs
- 4.9K
- Last 24h
- +41
- Trending rank
- #3803
- Security
- Pass
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/aws/agent-toolkit-for-aws --skill querying-aws-sagemaker-catalog- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads querying-aws-sagemaker-catalog automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
Security audits
Independent scans of the skill's instructions and bundled files.
SKILL.md
The instructions your agent reads when the skill is triggered.
Query AWS SageMaker Catalog System Tables
Overview
Works best with the AWS MCP server for sandboxed execution and audit logging. All commands below use the AWS CLI and work in any environment with configured AWS credentials.
Amazon SageMaker Unified Studio (whose catalog feature is referred to below as SageMaker Catalog) exports asset metadata as a daily-snapshot
Apache Iceberg table in the AWS-managed aws-sagemaker-catalog table bucket. This
enables SQL queries over your entire data catalog inventory — asset counts, governance
gaps, ownership audits, and historical comparisons — without building custom ETL.
Data is partitioned by snapshot_time and exported once daily (around midnight per
region). The table is read-only.
Decision Tree
| User intent | Use this skill? | Alternative |
|---|---|---|
| SQL analytics on catalog state (counts, governance, trends) | Yes | — |
| Historical comparison ("what changed in catalog last week") | Yes — time travel via snapshot_time | — |
| Find assets without owners or descriptions | Yes | — |
| Find a specific table by name or concept | No | finding-data-lake-assets or Glue Discovery search |
| Browse/enumerate catalog interactively | No | exploring-data-catalog |
| Run a query on a table's data | No | querying-data-lake |
| Manage catalog metadata (add descriptions, tags) | No | Glue Discovery put-form-type / associate-glossary-terms |
Common Tasks
1. Check If Configured
aws datazone get-data-export-configuration \
--domain-identifier <DOMAIN_ID> \
--region <REGION>
- If no domain exists:
aws datazone list-domains --region <REGION> - If export not enabled: guide user to enable.
- One domain per account per region.
Verify table bucket exists:
aws s3tables list-table-buckets --region <REGION> \
--query "tableBuckets[?name=='aws-sagemaker-catalog']"
2. Enable
With KMS encryption (recommended for production):
aws datazone put-data-export-configuration \
--domain-identifier <DOMAIN_ID> \
--region <REGION> \
--enable-export \
--encryption-configuration kmsKeyArn=<KMS_KEY_ARN>,sseAlgorithm=aws:kms
Note: Encryption cannot be changed after creation. Always specify KMS for sensitive catalog data.
Without encryption (for quick testing only):
aws datazone put-data-export-configuration \
--domain-identifier <DOMAIN_ID> \
--region <REGION> \
--enable-export
First data available within 24 hours. See: Exporting asset metadata
3. Verify Permissions for Querying
Requires:
- S3 Tables federated catalog registered in Glue (
s3tablescatalog) - Lake Formation SELECT + DESCRIBE grants on the table
Grant access:
aws lakeformation grant-permissions \
--principal DataLakePrincipalIdentifier=<ROLE_ARN> \
--resource '{"Table": {"CatalogId": "<ACCOUNT>:s3tablescatalog/aws-sagemaker-catalog", "DatabaseName": "asset_metadata", "Name": "asset"}}' \
--permissions DESCRIBE SELECT \
--region <REGION>
4. Query
Query syntax:
"s3tablescatalog/aws-sagemaker-catalog"."asset_metadata"."asset"
Constraints:
-
You MUST always filter by
snapshot_time— without it, the query scans all historical snapshots and returns duplicates -
You MUST confirm workgroup and output location before executing
-
Default to
DATE(snapshot_time) = CURRENT_DATEfor current state -
You SHOULD use the key columns documented in this skill to build queries. If you need the full schema, run
get-tablesonce:aws glue get-tables --catalog-id "<ACCOUNT>:s3tablescatalog/aws-sagemaker-catalog" --database-name "asset_metadata" --region <REGION>
Key columns:
| Column | What it holds | Usage |
|---|---|---|
snapshot_time | Partition key — daily snapshot timestamp | Always filter on this |
asset_id | Unique catalog asset identifier | Primary key for lookups |
resource_type_enum | GlueTable, RedshiftTable, S3Collection, etc. | Filter by asset type |
resource_id | ARN or native identifier | Cross-reference with source systems |
asset_name | Business-friendly name | Display, search |
resource_name | Technical name (table name, prefix) | Filtering |
business_description | Business context (NULL if not provided) | Governance gaps |
extended_metadata | map<string,string> — flexible key-value attributes | Use bracket notation: extended_metadata['owningEntityId'] |
asset_created_time | When asset first appeared in catalog | Growth analysis |
asset_updated_time | Last modification time | Freshness checks |
Current catalog state:
SELECT resource_type_enum, COUNT(*) as count
FROM "s3tablescatalog/aws-sagemaker-catalog"."asset_metadata"."asset"
WHERE DATE(snapshot_time) = CURRENT_DATE
GROUP BY resource_type_enum
ORDER BY count DESC;
Assets without business descriptions:
SELECT asset_name, resource_name, resource_type_enum, account_id
FROM "s3tablescatalog/aws-sagemaker-catalog"."asset_metadata"."asset"
WHERE DATE(snapshot_time) = CURRENT_DATE
AND business_description IS NULL;
Asset growth over last 30 days:
SELECT DATE(snapshot_time) as date, COUNT(*) as total_assets
FROM "s3tablescatalog/aws-sagemaker-catalog"."asset_metadata"."asset"
WHERE DATE(snapshot_time) >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY DATE(snapshot_time)
ORDER BY date DESC;
Time travel — compare current vs 7 days ago (new descriptions added):
SELECT t.asset_id, t.resource_name,
p.business_description as before,
t.business_description as now
FROM "s3tablescatalog/aws-sagemaker-catalog"."asset_metadata"."asset" t
JOIN "s3tablescatalog/aws-sagemaker-catalog"."asset_metadata"."asset" p
ON t.asset_id = p.asset_id
WHERE DATE(t.snapshot_time) = CURRENT_DATE
AND DATE(p.snapshot_time) = CURRENT_DATE - INTERVAL '7' DAY
AND p.business_description IS NULL
AND t.business_description IS NOT NULL;
Assets by owner:
SELECT extended_metadata['owningEntityId'] as owner, COUNT(*) as count
FROM "s3tablescatalog/aws-sagemaker-catalog"."asset_metadata"."asset"
WHERE DATE(snapshot_time) = CURRENT_DATE
AND extended_metadata['owningEntityId'] IS NOT NULL
GROUP BY extended_metadata['owningEntityId']
ORDER BY count DESC;
Filter by metadata form field:
SELECT *
FROM "s3tablescatalog/aws-sagemaker-catalog"."asset_metadata"."asset"
WHERE DATE(snapshot_time) = CURRENT_DATE
AND extended_metadata['<metadata-form-name>.<field-name>'] = '<field-value>';
Key Behaviors
- Daily snapshots — exported around midnight per region
- Always filter by
snapshot_time— without it you get all history (duplicates, slow) - One domain per account per region — to switch domains, delete config first
- No additional charge beyond S3 Tables storage + Athena queries
- Read-only — to update asset metadata, use Glue Discovery APIs or SageMaker Unified Studio
Troubleshooting
| Error | Cause | Fix |
|---|---|---|
aws-sagemaker-catalog bucket not found | Export not enabled | Run put-data-export-configuration --enable-export |
Empty results with CURRENT_DATE | First export hasn't run yet (takes up to 24h) | Wait; try yesterday's date |
AccessDenied on query | Missing Lake Formation grants | Grant SELECT + DESCRIBE on the table |
CATALOG_NOT_FOUND | S3 Tables not registered in Glue | Enable integration: S3 console > Table buckets > Enable integration |
| Duplicate rows in results | Missing snapshot_time filter | Add WHERE DATE(snapshot_time) = CURRENT_DATE |
extended_metadata key returns NULL | Key doesn't exist for that asset | Check available keys: SELECT DISTINCT key FROM ... CROSS JOIN UNNEST(map_keys(extended_metadata)) AS t(key) WHERE DATE(snapshot_time) = CURRENT_DATE |
| Cannot update export encryption | Encryption set at creation time only | Delete and recreate export config |
Security Considerations
Data sensitivity: Catalog metadata exposes organizational structure including asset names, ownership, account IDs, naming conventions, and internal resource identifiers. Treat query results as sensitive by default.
Encryption at rest: Always enable KMS encryption when creating the export configuration. Encryption cannot be changed after creation. Additionally, configure SSE-KMS on your Athena workgroup output bucket.
Least-privilege access: Grant Lake Formation SELECT + DESCRIBE only on the specific asset_metadata.asset table to roles that need catalog analytics. Avoid granting access to the entire aws-sagemaker-catalog bucket.
Audit trail: Enable CloudTrail logging for DataZone (PutDataExportConfiguration, GetDataExportConfiguration), Athena (StartQueryExecution, GetQueryResults), and S3 Tables API calls to track who queries catalog metadata.
Credential hygiene: Use IAM roles with temporary credentials for querying. Avoid long-lived access keys for users accessing catalog metadata. Scope down or rotate principals when access is no longer needed.
Additional Resources
FAQ
What does the querying-aws-sagemaker-catalog skill do?
Runs SQL analytics on SageMaker Catalog asset metadata tables exported as Apache Iceberg in S3 Tables. Covers governance queries, asset growth tracking, ownership audits, time-travel over catalog state, and metadata quality analysis. Applies when querying catalog inventory, finding assets without descriptions, comparing catalog snapshots, or auditing data ownership. Trigger phrases: catalog inventory SQL, how many assets, assets without descriptions, asset growth over time, who owns this data, catalog governance, data quality audit, catalog analytics.
How do I install querying-aws-sagemaker-catalog?
Run npx skills add https://github.com/aws/agent-toolkit-for-aws --skill querying-aws-sagemaker-catalog in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/querying-aws-sagemaker-catalog/ (personal) or .claude/skills/querying-aws-sagemaker-catalog/ (project).
Which AI agents can use querying-aws-sagemaker-catalog?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is querying-aws-sagemaker-catalog safe to install?
Security partners have reviewed it: Gen Agent Trust Hub passed; Socket passed; Snyk passed. As with any skill, read the SKILL.md and bundled scripts before installing.
Who maintains querying-aws-sagemaker-catalog?
It is published by aws in aws/agent-toolkit-for-aws, and it is an official skill from the maker of the technology it covers. It has 4,925 recorded installs.
More from aws/agent-toolkit-for-aws
aws-iam
aws/agent-toolkit-for-aws
Provides verified corrections for IAM behaviors that AI agents frequently get wrong — policy evaluation edge cases, trust policy gotchas, STS session limits, Organizations quirks, and SAML/MFA specifics. Also provides structured workflows for IAM role management and baseline policy generation from application source code or a Terraform plan JSON. Covers condition operator safety (ForAnyValue/ForAllValues with Null checks), bucket policy deny patterns (VPC endpoint restrictions, org paths), confused deputy protection, and service role creation for AWS services (Glue, CloudTrail, Lambda, ECS, etc.) with aws:SourceAccount/aws:SourceArn trust conditions. Applies when creating IAM roles, writing IAM or bucket policies, generating policies from application source code or a Terraform plan JSON, working with STS, Organizations, or condition operators, or any task needing a service or execution role. Does not cover non-IAM authorization like Cognito user-pool policies or app-level RBAC.
aws-serverless
aws/agent-toolkit-for-aws
Routes serverless requests to a compute form factor and the skill that owns it. Applies first to Lambda, API Gateway, EventBridge, Step Functions and event-driven work while the compute, orchestration or integration choice is open — API backends, webhooks, jobs, cron, queue/stream consumers, durable workflows, event routing, Step Functions vs Durable Functions, and Pipes vs native event source mappings for queues and streams. Decides Event Functions on-demand vs Managed Instances, Web Functions, MicroVMs, EventBridge rules/Scheduler/Pipes/bus, HTTP vs REST API. Owns shared Lambda references (cold starts, SnapStart, concurrency, event sources, function URLs, throttling, troubleshooting). Defers to the specialized skill once a request commits to one form factor or service, or describes its defining feature, even unnamed (durable checkpointing and automatic step retries, Firecracker sandbox, managed instances, Node.js server). Not for EC2, ECS/EKS/Fargate or Amplify hosting.
aws-observability
aws/agent-toolkit-for-aws
Builds, configures, debugs, and optimizes AWS observability - operator-symptom questions and detecting Omni vs classic CloudWatch. CloudWatch on an already-reporting service: Log Insights, metric/composite/anomaly alarms, custom metrics/EMF, dashboards, X-Ray/ADOT tracing, canaries, CloudTrail, Dynamic Instrumentation (live breakpoints/snapshots), the Application Signals service map, and fleet health views. CloudWatch Omni on an existing Space: SQL over logs and traces, PromQL over metrics, Omni dashboards, Omni alerts, context graph for root cause, programmatic/IaC access (API/SDK/CLI/CloudFormation), driving Omni from a coding agent or skills, and evaluating AI agent quality from traces - on-demand and online scoring of live traffic, readback, custom evaluators. For first-time setup - creating an Omni Space, granting access, ingestion, or ADOT instrumentation for Application Signals (ServiceEvents, CI/CD metadata) or Omni - use setting-up-cloudwatch-observability. Not for app logging
aws-cdk
aws/agent-toolkit-for-aws
Authors, deploys, and troubleshoots AWS infrastructure using CDK with TypeScript or Python. Covers best practices, stack architecture, and construct patterns. Applies when writing CDK constructs, bootstrapping environments, running cdk deploy/synth/diff, fixing CDK or CloudFormation errors, planning stack structure, importing existing resources, resolving drift, or refactoring stacks without resource replacement.
aws-containers
aws/agent-toolkit-for-aws
Builds and deploys containerized workloads on Elastic Kubernetes Service (EKS), Elastic Container Service (ECS), Fargate, and ECR (Elastic Container Registry). Covers general EKS knowledge, Karpenter, AWS Load Balancer Controller and leveraging various open source Kubernetes projects with EKS. Covers general ECS knowledge, task definitions, Fargate services, ECS Exec, ECS Express Mode and ECS Managed Instances. Covers general Elastic Beanstalk knowledge, Elastic Beanstalk configuration and platforms supported by Elastic Beanstalk. Covers general ECR knowledge, ECR repository setup and lifecycle policies. Includes recommending, enabling, and reading Amazon ECS Action Logs to troubleshoot control-plane failures (deployment rollback/circuit-breaker, task placement, scaling, task replacement). Applies when deploying, debugging, or optimizing containers on AWS. Should be used instead of relying on internal knowledge for these services.
aws-billing-and-cost-management
aws/agent-toolkit-for-aws
Analyze AWS costs, find savings, manage budgets, evaluate Savings Plans and Reserved Instances, right-size EC2/Lambda/RDS/EBS with Compute Optimizer, look up service pricing, query CUR with Athena, detect cost anomalies, scope costs to billing views, and monitor Free Tier usage. Triggers on: AWS bill, cost analysis, reduce spend, savings plan, reserved instance, right-size, budget alert, cost optimization, pricing, free tier, cost anomaly, CUR, cost audit, billing view, billing view ARN.
Popular devops & cloud skills
microsoft-foundry
microsoft/azure-skills
Build, deploy, evaluate, optimize, fine-tune, and manage Microsoft Foundry agents, models, and resources end to end. USE FOR: foundry, azd ai agent, azd provision/deploy, hosted agent scaffold/develop/run/deploy/troubleshoot, prompt agent create, managed harness agent create, GitHub Copilot harness agent, create agent, update agent, add tool to agent, invoke agent, agent insights, evaluate agent, batch eval, continuous eval, continuous monitoring, agent CI/CD, optimize prompt, improve prompt, optimize agent instructions, Agent Optimizer scaffold, dataset curation from traces, deploy model, model fine-tuning (SFT/DPO/RFT), Foundry project, RBAC, role assignment, permissions, quota, capacity, region, deployment failure, AI Services, create Foundry resource, knowledge index, customize deployment, onboard, availability, training-data, grader, distillation, large file upload. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure
azure-diagnostics
microsoft/azure-skills
Debug Azure production issues on Azure using AppLens, Azure Monitor, resource health, and safe triage. WHEN: debug production issues, troubleshoot app service, app service high CPU, app service deployment failure, troubleshoot container apps, container app 503 or target port mismatch, troubleshoot functions, function telemetry missing or stopped after a config/connection-string change, troubleshoot AKS, VM RDP, Linux SSH, VM black screen, can't connect to VM, reset VM password, NSG or firewall blocking, kubectl cannot connect, kube-system/CoreDNS failures, pod pending, crashloop, node not ready, upgrade failures, analyze logs, KQL, insights, image pull failures, cold start issues, health probe failures, resource health, root cause of errors, troubleshoot event hubs, troubleshoot service bus, messaging SDK error, AMQP connection failure, message lock lost, lock renewal during long-running processing, service bus dead letter.
azure-prepare
microsoft/azure-skills
Prepare azd-based Azure projects for deployment: generates azure.yaml, infrastructure (Bicep/Terraform), and Dockerfiles for the Azure Developer CLI (azd) workflow. USE ONLY when the user explicitly wants to use azd as the deployment tool, or the project already has an azure.yaml file. DO NOT USE FOR: non-azd deployments, Python App Service code-only deploys (use python-appservice-deploy), or cross-cloud migration (use azure-cloud-migrate). WHEN: prepare app for azd, create azure.yaml, set up azd infrastructure, modernize app for Azure with azd, deploy with azd, function app, timer trigger, service bus trigger, event-driven function, managed identity, generate Bicep, generate Terraform, create and deploy to Azure.
azure-ai
microsoft/azure-skills
Use for Azure AI: Search, Speech, OpenAI, Document Intelligence. Helps with search, vector/hybrid search, speech-to-text, text-to-speech, transcription, OCR. WHEN: AI Search, query search, vector search, hybrid search, semantic search, speech-to-text, text-to-speech, transcribe, OCR, convert text to speech.
azure-deploy
microsoft/azure-skills
Execute Azure deployments for ALREADY-PREPARED applications that have existing .azure/deployment-plan.md and infrastructure files. DO NOT use this skill when the user asks to CREATE a new application — use azure-prepare instead. This skill runs azd up, azd deploy, terraform apply, and az deployment commands with built-in error recovery. Requires .azure/deployment-plan.md from azure-prepare and validated status from azure-validate. WHEN: "run azd up", "run azd deploy", "execute deployment", "push to production", "push to cloud", "go live", "ship it", "bicep deploy", "terraform apply", "publish to Azure", "launch on Azure". DO NOT USE WHEN: "create and deploy", "build and deploy", "create a new app", "set up infrastructure", "create and deploy to Azure using Terraform" — use azure-prepare for these.
azure-validate
microsoft/azure-skills
Pre-deployment validation for Azure readiness. Run deep checks on configuration, infrastructure (Bicep or Terraform), RBAC role assignments, managed identity permissions, and prerequisites before deploying. WHEN: validate my app, check deployment readiness, run preflight checks, verify configuration, check if ready to deploy, validate azure.yaml, validate Bicep, test before deploying, troubleshoot deployment errors, validate Azure Functions, validate function app, validate serverless deployment, verify RBAC roles, check role assignments, review managed identity permissions, what-if analysis, validate Container Apps deployment.