querying-data-lake
Execute and manage Athena SQL queries across default and federated catalogs (Glue, S3 Tables, Redshift). Triggers on phrases like: query data, run SQL, athena query, analyze table, SQL query, workgroup status, profile table, query Redshift catalog, query S3 Tables. Do NOT use for finding specific data assets (use finding-data-lake-assets), full catalog audits (use exploring-data-catalog), importing data (use ingesting-into-data-lake).
- Installs
- 6.7K
- Last 24h
- +41
- Trending rank
- #3779
- Security
- Pass
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/aws/agent-toolkit-for-aws --skill querying-data-lake- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads querying-data-lake automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
Security audits
Independent scans of the skill's instructions and bundled files.
SKILL.md
The instructions your agent reads when the skill is triggered.
Query Data Lake
Execute SQL queries on Amazon Athena across default and federated catalogs (Glue, S3 Tables, Redshift) with workgroup selection, statement classification, and error recovery.
Overview
Executes and manages Athena SQL queries across default and federated catalogs. Selects a workgroup, resolves target assets (delegating fuzzy references to finding-data-lake-assets), classifies statements for safety, and reports cost and data scanned. Use the AWS MCP server for sandboxed execution and audit logging; the same AWS CLI commands work directly when the MCP server is not available.
Constraints for parameter acquisition:
- You MUST accept a single optional argument: SQL text, a named-query name, a workgroup name, a catalog name, or
profile TABLE_NAME - You MUST accept the argument as direct text or a pointer to a file containing SQL
- You MUST ask the user for the target AWS region if not already set
- You MUST confirm the output S3 location before executing any non-trivial query
- You MUST respect the user's decision to abort at any step
Common Tasks
1. Verify Dependencies
Check for required tools and AWS access before running queries.
Constraints:
- You MUST verify AWS MCP server tools are available (
aws___call_aws) and run queries through them when present; fall back to AWS CLI only if the MCP server is unavailable - You MUST NOT fall back to shell or Bash for query execution — results must be captured via the MCP tool or
aws athenaCLI so output location and cost are tracked - You MUST confirm credentials with
aws sts get-caller-identityand inform the user about any missing tools
2. Resolve Workgroup
Check caller identity, list workgroups, auto-select the best one (see workgroup-selection.md).
Constraints:
- You MUST select a workgroup before submitting any query (prevents output-location errors)
- You MUST present the selected workgroup and its output location to the user
- You MUST NOT auto-escalate to a different workgroup on failure without user confirmation
3. Resolve the Target Asset
If the user refers to a table by name, by business concept ("our quarterly report", "the sales data"), by S3 path, or by catalog without specifying the table, delegate to finding-data-lake-assets to return the concrete database.table (and catalog if non-default).
Constraints:
- You MUST NOT attempt to resolve fuzzy asset references with
athena list-data-catalogsor by iteratingget-tables— those miss federated catalogs and waste tokens - You SHOULD skip this step only when the user provides a fully-qualified reference (exact
database.table) or raw SQL they want executed as-is - You MUST state the resolved asset explicitly before building the query: "Found [table] in [catalog]. Using this for the query."
- You SHOULD default to the default Glue catalog unless the user mentions "federated", "Redshift", "S3 Tables", or
finding-data-lake-assetsreturns a different catalog
4. Discover Schema
For analytical queries, You SHOULD profile the target table before building the final query. You MUST show sample rows (SELECT ... LIMIT 5) as part of profiling.
5. Build Query
Table addressing depends on catalog type:
- Default Glue catalog:
database.table(omit the catalog prefix for single-catalog queries). In cross-catalog queries, qualify default-catalog tables with"awsdatacatalog".database.table. - Registered data source:
datasource.database.table - Unregistered Glue catalog:
"catalog/subcatalog".database.table
6. Classify and Execute
Classify the SQL statement before executing:
| Statement | Behavior |
|---|---|
SELECT, SHOW, DESCRIBE, EXPLAIN | Safe — execute |
INSERT, UPDATE, DELETE, DROP, ALTER, CREATE, TRUNCATE, MERGE | Destructive — warn the user and require explicit confirmation |
| Unsure | Treat as destructive; confirm |
Example tool call (via AWS MCP server):
aws___call_aws(command="aws athena start-query-execution --work-group <WORKGROUP_NAME> --query-string '<sql>' --query-execution-context Database=<db>")
For federated or S3 Tables catalogs, also set Catalog=<CATALOG_PATH> in the execution context (e.g. Catalog=s3tablescatalog/<BUCKET_NAME>).
Constraints:
- You MUST warn the user before executing when the target is Redshift-federated ("No partition pruning — every query scans the full table")
- You MUST warn the user before executing a cross-catalog join ("Cross-catalog joins incur network overhead and may be slow")
- You MUST confirm the output S3 location before executing
- You MUST explain which tool is being called before executing
- You MUST respect the user's decision to abort
7. Present and Recover
Present results with cost, data scanned, duration, and actionable insights. On failure, list available workgroups and let the user choose which to retry with.
Argument Routing
Resolve in this order; stop at the first match:
- Contains SQL keywords (
SELECT,SHOW,DESCRIBE,INSERT, etc.) — SQL text, execute directly profile TABLE_NAME— run comprehensive table profiling (see query-patterns.md)- Matches a known named query — look up and execute
- Matches a known workgroup — show workgroup status and recent queries
- Matches a known catalog — delegate to
exploring-data-catalogto enumerate databases and tables - No args — show recent query activity and available tables
Principles
- Always select workgroup before executing (prevents output-location errors)
- Profile unfamiliar tables before running analytical queries
- Present cost alongside results so users build cost awareness
- Suggest
LIMITfor exploratory queries on large tables - Never ask domain questions with obvious answers, but always confirm security-relevant actions (workgroup switches, output location changes, non-SELECT statements)
Troubleshooting
| Error | Cause | Fix |
|---|---|---|
| Redshift identifier error with mixed case | Redshift-federated names are lowercase only | Lowercase the identifier |
CatalogId validation failure | ARN passed instead of catalog name | Pass the catalog name, not the ARN |
Cross-catalog information_schema returns nothing | Missing catalog qualifier | Use catalog-qualified path: "catalog".information_schema.tables |
| Query fails with output-location error | Workgroup has no output location configured | Select a different workgroup with an output location, or configure one |
| Destructive statement executed without confirmation | Statement classification skipped | Always classify INSERT/UPDATE/DELETE/DROP/ALTER/CREATE/TRUNCATE/MERGE and confirm with the user |
Additional Resources
- Workgroup selection logic
- Common query patterns
- Athena best practices
- Athena federated query
FAQ
What does the querying-data-lake skill do?
Execute and manage Athena SQL queries across default and federated catalogs (Glue, S3 Tables, Redshift). Triggers on phrases like: query data, run SQL, athena query, analyze table, SQL query, workgroup status, profile table, query Redshift catalog, query S3 Tables. Do NOT use for finding specific data assets (use finding-data-lake-assets), full catalog audits (use exploring-data-catalog), importing data (use ingesting-into-data-lake).
How do I install querying-data-lake?
Run npx skills add https://github.com/aws/agent-toolkit-for-aws --skill querying-data-lake in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/querying-data-lake/ (personal) or .claude/skills/querying-data-lake/ (project).
Which AI agents can use querying-data-lake?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is querying-data-lake safe to install?
Security partners have reviewed it: Gen Agent Trust Hub passed; Socket passed; Snyk passed. As with any skill, read the SKILL.md and bundled scripts before installing.
Who maintains querying-data-lake?
It is published by aws in aws/agent-toolkit-for-aws, and it is an official skill from the maker of the technology it covers. It has 6,680 recorded installs.
More from aws/agent-toolkit-for-aws
aws-iam
aws/agent-toolkit-for-aws
Provides verified corrections for IAM behaviors that AI agents frequently get wrong — policy evaluation edge cases, trust policy gotchas, STS session limits, Organizations quirks, and SAML/MFA specifics. Also provides structured workflows for IAM role management and baseline policy generation from application source code or a Terraform plan JSON. Covers condition operator safety (ForAnyValue/ForAllValues with Null checks), bucket policy deny patterns (VPC endpoint restrictions, org paths), confused deputy protection, and service role creation for AWS services (Glue, CloudTrail, Lambda, ECS, etc.) with aws:SourceAccount/aws:SourceArn trust conditions. Applies when creating IAM roles, writing IAM or bucket policies, generating policies from application source code or a Terraform plan JSON, working with STS, Organizations, or condition operators, or any task needing a service or execution role. Does not cover non-IAM authorization like Cognito user-pool policies or app-level RBAC.
aws-serverless
aws/agent-toolkit-for-aws
Routes serverless requests to a compute form factor and the skill that owns it. Applies first to Lambda, API Gateway, EventBridge, Step Functions and event-driven work while the compute, orchestration or integration choice is open — API backends, webhooks, jobs, cron, queue/stream consumers, durable workflows, event routing, Step Functions vs Durable Functions, and Pipes vs native event source mappings for queues and streams. Decides Event Functions on-demand vs Managed Instances, Web Functions, MicroVMs, EventBridge rules/Scheduler/Pipes/bus, HTTP vs REST API. Owns shared Lambda references (cold starts, SnapStart, concurrency, event sources, function URLs, throttling, troubleshooting). Defers to the specialized skill once a request commits to one form factor or service, or describes its defining feature, even unnamed (durable checkpointing and automatic step retries, Firecracker sandbox, managed instances, Node.js server). Not for EC2, ECS/EKS/Fargate or Amplify hosting.
aws-observability
aws/agent-toolkit-for-aws
Builds, configures, debugs, and optimizes AWS observability - operator-symptom questions and detecting Omni vs classic CloudWatch. CloudWatch on an already-reporting service: Log Insights, metric/composite/anomaly alarms, custom metrics/EMF, dashboards, X-Ray/ADOT tracing, canaries, CloudTrail, Dynamic Instrumentation (live breakpoints/snapshots), the Application Signals service map, and fleet health views. CloudWatch Omni on an existing Space: SQL over logs and traces, PromQL over metrics, Omni dashboards, Omni alerts, context graph for root cause, programmatic/IaC access (API/SDK/CLI/CloudFormation), driving Omni from a coding agent or skills, and evaluating AI agent quality from traces - on-demand and online scoring of live traffic, readback, custom evaluators. For first-time setup - creating an Omni Space, granting access, ingestion, or ADOT instrumentation for Application Signals (ServiceEvents, CI/CD metadata) or Omni - use setting-up-cloudwatch-observability. Not for app logging
aws-cdk
aws/agent-toolkit-for-aws
Authors, deploys, and troubleshoots AWS infrastructure using CDK with TypeScript or Python. Covers best practices, stack architecture, and construct patterns. Applies when writing CDK constructs, bootstrapping environments, running cdk deploy/synth/diff, fixing CDK or CloudFormation errors, planning stack structure, importing existing resources, resolving drift, or refactoring stacks without resource replacement.
aws-containers
aws/agent-toolkit-for-aws
Builds and deploys containerized workloads on Elastic Kubernetes Service (EKS), Elastic Container Service (ECS), Fargate, and ECR (Elastic Container Registry). Covers general EKS knowledge, Karpenter, AWS Load Balancer Controller and leveraging various open source Kubernetes projects with EKS. Covers general ECS knowledge, task definitions, Fargate services, ECS Exec, ECS Express Mode and ECS Managed Instances. Covers general Elastic Beanstalk knowledge, Elastic Beanstalk configuration and platforms supported by Elastic Beanstalk. Covers general ECR knowledge, ECR repository setup and lifecycle policies. Includes recommending, enabling, and reading Amazon ECS Action Logs to troubleshoot control-plane failures (deployment rollback/circuit-breaker, task placement, scaling, task replacement). Applies when deploying, debugging, or optimizing containers on AWS. Should be used instead of relying on internal knowledge for these services.
aws-billing-and-cost-management
aws/agent-toolkit-for-aws
Analyze AWS costs, find savings, manage budgets, evaluate Savings Plans and Reserved Instances, right-size EC2/Lambda/RDS/EBS with Compute Optimizer, look up service pricing, query CUR with Athena, detect cost anomalies, scope costs to billing views, and monitor Free Tier usage. Triggers on: AWS bill, cost analysis, reduce spend, savings plan, reserved instance, right-size, budget alert, cost optimization, pricing, free tier, cost anomaly, CUR, cost audit, billing view, billing view ARN.
Popular data & databases skills
supabase-postgres-best-practices
supabase/agent-skills
Postgres best practices maintained by Supabase, for Postgres running anywhere. Load this skill BEFORE writing or changing anything that lives in a Postgres database: creating or altering tables and columns (including choosing column types), schema design, migrations and declarative schema files, RLS policies and the tests that verify them, indexes, triggers, database functions, queues and scheduled jobs (pgcron, pgmq), vector/semantic search (pgvector), and restoring dumps (pgrestore) or importing data. Also load it when diagnosing slow queries, high CPU, timeouts, EXPLAIN plans, connection exhaustion, locking, bloat, or rows visible to the wrong user or tenant. This is not just a performance guide — schema, migration, security, and SQL authoring tasks need these rules too, even for a one-column change or a single query.
prisma-client-api
prisma/skills
Prisma Client API reference covering model queries, filters, operators, and client methods. Use when writing database queries, using CRUD operations, filtering data, or configuring Prisma Client. Triggers on "prisma query", "findMany", "create", "update", "delete", "$transaction".
prisma-database-setup
prisma/skills
Deprecated compatibility name for prisma-orm-setup. Use only when a user or existing integration explicitly invokes prisma-database-setup; otherwise select prisma-orm-setup.
prisma-cli
prisma/skills
Prisma ORM 7 CLI reference for existing Prisma 7 applications or explicitly selected Prisma 7 workflows. Covers prisma init, generate, migrate, db, dev, studio, and mcp. For setup, connection repair, or Prisma 8, use prisma-orm-setup; not the Prisma Platform CLI.
prisma-postgres
prisma/skills
Deprecated compatibility name for prisma-postgres-setup. Use only when a user or existing integration explicitly invokes prisma-postgres; otherwise select prisma-postgres-setup.
prisma-driver-adapter-implementation
prisma/skills
Required reference for Prisma ORM 7 SQL driver adapter work. Use when implementing or modifying adapters, adding database drivers, or touching SqlDriverAdapter, Transaction, savepoint, result mapping, or DriverAdapterError behavior. Covers current transaction lifecycle, optional savepoint hooks, original database-error preservation, and verification.