querying-aws-s3

Q: How do I install querying-aws-s3?

Run npx skills add https://github.com/aws/agent-toolkit-for-aws --skill querying-aws-s3 in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/querying-aws-s3/ (personal) or .claude/skills/querying-aws-s3/ (project).

by aws OfficialDevOps & Cloud

Queries S3 object metadata, tracks bucket activity, audits object changes, searches annotations, and analyzes storage metrics using S3 Metadata system tables (journal, inventory, annotation) and S3 Storage Lens tables via Athena SQL. Applies when counting objects, finding recent uploads or deletions, identifying who wrote to a prefix, breaking down storage classes, finding objects by tag, searching annotation content, analyzing storage lens metrics, or enabling S3 Metadata tracking. Prefers system tables over raw S3 APIs (list-objects-v2, head-object) at scale. Trigger phrases: bucket activity, object count, who uploaded, track deletions, storage class breakdown, find by tag, search annotations, storage lens metrics, audit bucket changes.

Installs: 5.6K
Last 24h: +41
Trending rank: #3807
Security: Pass

Install

Works with every agent the skills CLI supports.

npx skills add https://github.com/aws/agent-toolkit-for-aws --skill querying-aws-s3

01
Install
Run the command in your project. The CLI asks which agents to install for.
02
Describe the task
Ask your agent for something this skill covers. It loads querying-aws-s3 automatically when relevant.
03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.

Security audits

Independent scans of the skill's instructions and bundled files.

Gen Agent Trust Hub PassSAFE ·This skill provides templates and instructions for querying Amazon S3 Metadata and Storage Lens tables. It emphasizes security best practices, including encryption and least-privilege IAM policies, while utilizing standard AWS CLI tools.

Socket PassNo alerts

Snyk PassLOW ·Risk: LOW · No issues

SKILL.md

The instructions your agent reads when the skill is triggered.

Query AWS S3 System Tables

Overview

Works best with the AWS MCP server for sandboxed execution and audit logging. All commands below use the AWS CLI and work in any environment with configured AWS credentials. Use IAM roles or temporary credentials; avoid long-lived access keys.

Amazon S3 Metadata provides continuously-updated Apache Iceberg tables that capture object-level metadata for general-purpose buckets. S3 Storage Lens exports aggregated storage and activity metrics as Iceberg tables. Both are read-only, stored in the AWS-managed aws-s3 table bucket, and queryable via Amazon Athena.

System tables are preferred over raw S3 APIs (list-objects-v2, head-object) because:

list-objects-v2 paginates at 1000 objects/page — inefficient for large buckets (millions or billions of objects). The inventory table answers SELECT COUNT(*) in seconds at any scale.
list-objects-v2 cannot identify who uploaded an object, from which IP, or when something was deleted. Only the journal table has requester, source_ip_address, and delete event tracking.
Filtering by tag requires get-object-tagging per object. The inventory table has object_tags as a queryable map column.

Decision Tree

User intent	Use this skill?	Table	Alternative
How many objects in my bucket	Yes	inventory	—
What was recently uploaded/deleted	Yes	journal	—
Who wrote/deleted objects (audit)	Yes	journal (requester, source_ip)	—
Storage class breakdown	Yes	inventory	—
Find objects by tag or user metadata	Yes	inventory	—
Search annotation content	Yes	annotation	Single object → direct API `get-object-annotation`
Write/update an annotation	No	—	Direct API: `put-object-annotation` (tables are read-only)
Query data inside objects	No	—	`querying-data-lake`
Bucket-level storage metrics/trends	Yes	Storage Lens tables	—
Enable metadata tracking	Yes	see Enable section	—

Common Tasks

1. Check If Configured

Before querying, confirm S3 Metadata is enabled on the target bucket.

aws s3api get-bucket-metadata-configuration --bucket <BUCKET> --region <REGION>

Interpret the response:

MetadataConfigurationNotFound error → not enabled. See Enable section below.
TableStatus: ACTIVE → ready to query.
TableStatus: BACKFILLING → queryable but inventory may be incomplete.
TableStatus: FAILED → check error field (usually IAM).

For Storage Lens:

aws s3control get-storage-lens-configuration --account-id <ACCOUNT> --config-id <CONFIG_ID> --region <REGION>

Look for DataExport.StorageLensTableDestination.IsEnabled: true.

2. Enable (if not configured)

Enable S3 Metadata on a bucket:

aws s3api create-bucket-metadata-configuration \
  --bucket <BUCKET> \
  --region <REGION> \
  --metadata-configuration '{
    "JournalTableConfiguration": {"RecordExpiration": {"Expiration": "DISABLED"}},
    "InventoryTableConfiguration": {"ConfigurationState": "ENABLED"}
  }'

To also enable annotations (requires a service role):

aws s3api create-bucket-metadata-configuration \
  --bucket <BUCKET> \
  --region <REGION> \
  --metadata-configuration '{
    "JournalTableConfiguration": {"RecordExpiration": {"Expiration": "ENABLED", "Days": 90}},
    "InventoryTableConfiguration": {"ConfigurationState": "ENABLED"},
    "AnnotationTableConfiguration": {"ConfigurationState": "ENABLED", "Role": "<ROLE_ARN>"}
  }'

Enable Storage Lens S3 Tables export:

aws s3control put-storage-lens-configuration \
  --account-id <ACCOUNT> \
  --config-id <CONFIG_ID> \
  --region <REGION> \
  --storage-lens-configuration '{
    "Id": "<CONFIG_ID>",
    "IsEnabled": true,
    "AccountLevel": {"BucketLevel": {}},
    "DataExport": {
      "StorageLensTableDestination": {"IsEnabled": true}
    }
  }'

Register S3 Tables federated catalog in Glue (required for Athena access):

aws glue create-catalog --region <REGION> --cli-input-json '{
  "Name": "s3tablescatalog",
  "CatalogInput": {
    "FederatedCatalog": {
      "Identifier": "arn:aws:s3tables:<REGION>:<ACCOUNT>:bucket/*",
      "ConnectionName": "aws:s3tables"
    }
  }
}'

For setup permissions and IAM role requirements, see Security Considerations below.

3. Verify Permissions

Querying requires:

Athena execution permissions
S3 Tables read permissions (see least-privilege policy in Security Considerations)
The S3 Tables federated catalog registered in Glue (s3tablescatalog)
Athena workgroup with SSE-KMS encryption configured on the output location

If CATALOG_NOT_FOUND errors occur, the Glue integration may not be enabled. See: Integrating S3 Tables with AWS analytics services

4. Identify the Target Table

S3 Metadata tables — namespace is b_<bucket-name>:

Table	What it captures
`journal`	Event log — every CREATE, DELETE, UPDATE_METADATA, and annotation events. Near real-time.
`inventory`	Current state — one row per object (latest version). Updates within 1 hour.
`annotation`	Annotation payloads — `text_value` column holds the full content. Near real-time.

Storage Lens tables — namespace is lens_<config-id>_exp:

Table	What it captures
`default_storage_metrics`	Per-bucket/prefix: object count, size, storage class breakdown. Daily.
`default_activity_metrics`	Per-bucket/prefix: GET/PUT/DELETE request counts. Daily.
`bucket_property_metrics`	Bucket config: versioning, encryption, lifecycle settings. Daily.

5. Query

Query syntax:

"s3tablescatalog/aws-s3"."<namespace>"."<table>"

Constraints:

You MUST confirm workgroup and output location before executing
You MUST ensure the Athena workgroup enforces SSE-KMS encryption on query results
You MUST warn user that tables are read-only — no INSERT/UPDATE/DELETE
You SHOULD use the key columns documented in this skill to build queries. If you need the full schema (e.g., AWS has added new columns), run get-tables once on any single namespace — schemas are identical across all instances of the same table type:
```
aws glue get-tables --catalog-id "<ACCOUNT>:s3tablescatalog/aws-s3" --database-name "<namespace>" --region <REGION>
```

Journal — audit who changed what:

SELECT key, record_type, record_timestamp, requester, source_ip_address
FROM "s3tablescatalog/aws-s3"."b_<bucket>"."journal"
WHERE record_type = 'DELETE'
  AND record_timestamp > current_timestamp - interval '24' hour
ORDER BY record_timestamp DESC;

Journal — track annotation events:

SELECT key, record_type, annotation.name, record_timestamp
FROM "s3tablescatalog/aws-s3"."b_<bucket>"."journal"
WHERE record_type IN ('CREATE_ANNOTATION', 'DELETE_ANNOTATION', 'UPDATE_ANNOTATION_METADATA')
ORDER BY record_timestamp DESC LIMIT 20;

Inventory — find objects by storage class:

SELECT key, size, storage_class, last_modified_date
FROM "s3tablescatalog/aws-s3"."b_<bucket>"."inventory"
WHERE storage_class = 'GLACIER'
ORDER BY size DESC LIMIT 50;

Inventory — find objects by tag:

SELECT key, size, object_tags
FROM "s3tablescatalog/aws-s3"."b_<bucket>"."inventory"
WHERE object_tags['environment'] = 'staging';

Annotation — search across payloads:

SELECT object_key, name, text_value
FROM "s3tablescatalog/aws-s3"."b_<bucket>"."annotation"
WHERE text_value LIKE '%error%';

Annotation — extract JSON fields:

SELECT object_key, json_extract_scalar(text_value, '$.status') as status
FROM "s3tablescatalog/aws-s3"."b_<bucket>"."annotation"
WHERE name = 'pipeline_status'
  AND json_extract_scalar(text_value, '$.status') = 'FAILED';

Storage Lens — storage distribution:

SELECT *
FROM "s3tablescatalog/aws-s3"."lens_<config-id>_exp"."default_storage_metrics"
LIMIT 20;

Routing: Athena vs Direct API

Scenario	Use
Single known object + annotation name	Direct API: `get-object-annotation`
Aggregate/count across many objects	Athena on annotation or inventory table
Full-text search across annotation payloads	Athena with `LIKE` or `json_extract_scalar`
Write/update an annotation	Direct API: `put-object-annotation` (table is read-only)
Feature not configured on bucket	Direct API loop (`list-objects-v2` + `head-object`); suggest enabling S3 Metadata

Troubleshooting

Error	Cause	Fix
`CATALOG_NOT_FOUND`	S3 Tables not registered in Glue	Enable integration: S3 console > Table buckets > Enable integration
Empty results from journal	Feature just enabled; no events recorded yet	Upload/delete an object and wait ~1 minute
Empty results from inventory	Table still `BACKFILLING`	Check status; wait for ACTIVE (minutes to hours depending on object count)
`AccessDenied` querying table	Missing `s3tables:GetTable` or `GetTableMetadataLocation`	See Security Considerations below
Wrong namespace	Bucket name has periods	Periods are converted to underscores in namespace: `my.bucket` → `b_my_bucket`
No Storage Lens data	First delivery takes up to 48 hours	Wait; no historical backfill

Security Considerations

Least-Privilege IAM Policy

Scope permissions to specific table bucket ARNs rather than using wildcards:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3tables:GetTable",
        "s3tables:GetTableMetadataLocation",
        "s3tables:GetTableData",
        "s3tables:GetNamespace",
        "s3tables:ListTables",
        "s3tables:ListNamespaces",
        "s3tables:GetTableBucket"
      ],
      "Resource": [
        "arn:aws:s3tables:<REGION>:<ACCOUNT>:bucket/aws-s3",
        "arn:aws:s3tables:<REGION>:<ACCOUNT>:bucket/aws-s3/*"
      ]
    }
  ]
}

Data Sensitivity

Journal query results may contain sensitive fields:

requester — AWS account ID or service principal that made the request
source_ip_address — IP address of the requester

Query results containing these fields should be stored in encrypted, access-controlled locations. Avoid logging or sharing raw query output that contains IP addresses or principal identifiers.

Encryption for Query Results

Configure the Athena workgroup with EncryptionConfiguration to encrypt query results at rest:

{
  "ResultConfiguration": {
    "EncryptionConfiguration": {
      "EncryptionOption": "SSE_KMS",
      "KmsKey": "arn:aws:kms:<REGION>:<ACCOUNT>:key/<KEY_ID>"
    }
  }
}

Audit Trail

Enable CloudTrail logging for Athena (StartQueryExecution, GetQueryResults) and S3 Tables (s3tables:GetTableData) API calls to maintain an audit trail of who queried what metadata. Ensure CloudTrail logs are encrypted with SSE-KMS and stored in a bucket with access logging enabled.

Additional Resources

FAQ

What does the querying-aws-s3 skill do?

How do I install querying-aws-s3?

Run npx skills add https://github.com/aws/agent-toolkit-for-aws --skill querying-aws-s3 in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/querying-aws-s3/ (personal) or .claude/skills/querying-aws-s3/ (project).

Which AI agents can use querying-aws-s3?

Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.

Is querying-aws-s3 safe to install?

Security partners have reviewed it: Gen Agent Trust Hub passed; Socket passed; Snyk passed. As with any skill, read the SKILL.md and bundled scripts before installing.

Who maintains querying-aws-s3?

It is published by aws in aws/agent-toolkit-for-aws, and it is an official skill from the maker of the technology it covers. It has 5,587 recorded installs.

More from aws/agent-toolkit-for-aws

View all →

aws-iam

aws/agent-toolkit-for-aws

Provides verified corrections for IAM behaviors that AI agents frequently get wrong — policy evaluation edge cases, trust policy gotchas, STS session limits, Organizations quirks, and SAML/MFA specifics. Also provides structured workflows for IAM role management and baseline policy generation from application source code or a Terraform plan JSON. Covers condition operator safety (ForAnyValue/ForAllValues with Null checks), bucket policy deny patterns (VPC endpoint restrictions, org paths), confused deputy protection, and service role creation for AWS services (Glue, CloudTrail, Lambda, ECS, etc.) with aws:SourceAccount/aws:SourceArn trust conditions. Applies when creating IAM roles, writing IAM or bucket policies, generating policies from application source code or a Terraform plan JSON, working with STS, Organizations, or condition operators, or any task needing a service or execution role. Does not cover non-IAM authorization like Cognito user-pool policies or app-level RBAC.

OfficialDevOps & Cloud10.3K installs

aws-serverless

aws/agent-toolkit-for-aws

Routes serverless requests to a compute form factor and the skill that owns it. Applies first to Lambda, API Gateway, EventBridge, Step Functions and event-driven work while the compute, orchestration or integration choice is open — API backends, webhooks, jobs, cron, queue/stream consumers, durable workflows, event routing, Step Functions vs Durable Functions, and Pipes vs native event source mappings for queues and streams. Decides Event Functions on-demand vs Managed Instances, Web Functions, MicroVMs, EventBridge rules/Scheduler/Pipes/bus, HTTP vs REST API. Owns shared Lambda references (cold starts, SnapStart, concurrency, event sources, function URLs, throttling, troubleshooting). Defers to the specialized skill once a request commits to one form factor or service, or describes its defining feature, even unnamed (durable checkpointing and automatic step retries, Firecracker sandbox, managed instances, Node.js server). Not for EC2, ECS/EKS/Fargate or Amplify hosting.

OfficialDevOps & Cloud10.2K installs

aws-observability

aws/agent-toolkit-for-aws

Builds, configures, debugs, and optimizes AWS observability - operator-symptom questions and detecting Omni vs classic CloudWatch. CloudWatch on an already-reporting service: Log Insights, metric/composite/anomaly alarms, custom metrics/EMF, dashboards, X-Ray/ADOT tracing, canaries, CloudTrail, Dynamic Instrumentation (live breakpoints/snapshots), the Application Signals service map, and fleet health views. CloudWatch Omni on an existing Space: SQL over logs and traces, PromQL over metrics, Omni dashboards, Omni alerts, context graph for root cause, programmatic/IaC access (API/SDK/CLI/CloudFormation), driving Omni from a coding agent or skills, and evaluating AI agent quality from traces - on-demand and online scoring of live traffic, readback, custom evaluators. For first-time setup - creating an Omni Space, granting access, ingestion, or ADOT instrumentation for Application Signals (ServiceEvents, CI/CD metadata) or Omni - use setting-up-cloudwatch-observability. Not for app logging

OfficialDevOps & Cloud10K installs

aws-cdk

aws/agent-toolkit-for-aws

Authors, deploys, and troubleshoots AWS infrastructure using CDK with TypeScript or Python. Covers best practices, stack architecture, and construct patterns. Applies when writing CDK constructs, bootstrapping environments, running cdk deploy/synth/diff, fixing CDK or CloudFormation errors, planning stack structure, importing existing resources, resolving drift, or refactoring stacks without resource replacement.

OfficialDevOps & Cloud9.5K installs

aws-containers

aws/agent-toolkit-for-aws

Builds and deploys containerized workloads on Elastic Kubernetes Service (EKS), Elastic Container Service (ECS), Fargate, and ECR (Elastic Container Registry). Covers general EKS knowledge, Karpenter, AWS Load Balancer Controller and leveraging various open source Kubernetes projects with EKS. Covers general ECS knowledge, task definitions, Fargate services, ECS Exec, ECS Express Mode and ECS Managed Instances. Covers general Elastic Beanstalk knowledge, Elastic Beanstalk configuration and platforms supported by Elastic Beanstalk. Covers general ECR knowledge, ECR repository setup and lifecycle policies. Includes recommending, enabling, and reading Amazon ECS Action Logs to troubleshoot control-plane failures (deployment rollback/circuit-breaker, task placement, scaling, task replacement). Applies when deploying, debugging, or optimizing containers on AWS. Should be used instead of relying on internal knowledge for these services.

OfficialDevOps & Cloud9.5K installs

aws-billing-and-cost-management

aws/agent-toolkit-for-aws

Analyze AWS costs, find savings, manage budgets, evaluate Savings Plans and Reserved Instances, right-size EC2/Lambda/RDS/EBS with Compute Optimizer, look up service pricing, query CUR with Athena, detect cost anomalies, scope costs to billing views, and monitor Free Tier usage. Triggers on: AWS bill, cost analysis, reduce spend, savings plan, reserved instance, right-size, budget alert, cost optimization, pricing, free tier, cost anomaly, CUR, cost audit, billing view, billing view ARN.

OfficialDevOps & Cloud9K installs

Popular devops & cloud skills

View category →

microsoft-foundry

microsoft/azure-skills

Build, deploy, evaluate, optimize, fine-tune, and manage Microsoft Foundry agents, models, and resources end to end. USE FOR: foundry, azd ai agent, azd provision/deploy, hosted agent scaffold/develop/run/deploy/troubleshoot, prompt agent create, managed harness agent create, GitHub Copilot harness agent, create agent, update agent, add tool to agent, invoke agent, agent insights, evaluate agent, batch eval, continuous eval, continuous monitoring, agent CI/CD, optimize prompt, improve prompt, optimize agent instructions, Agent Optimizer scaffold, dataset curation from traces, deploy model, model fine-tuning (SFT/DPO/RFT), Foundry project, RBAC, role assignment, permissions, quota, capacity, region, deployment failure, AI Services, create Foundry resource, knowledge index, customize deployment, onboard, availability, training-data, grader, distillation, large file upload. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure

OfficialDevOps & Cloud579.1K installs

azure-diagnostics

microsoft/azure-skills

Debug Azure production issues on Azure using AppLens, Azure Monitor, resource health, and safe triage. WHEN: debug production issues, troubleshoot app service, app service high CPU, app service deployment failure, troubleshoot container apps, container app 503 or target port mismatch, troubleshoot functions, function telemetry missing or stopped after a config/connection-string change, troubleshoot AKS, VM RDP, Linux SSH, VM black screen, can't connect to VM, reset VM password, NSG or firewall blocking, kubectl cannot connect, kube-system/CoreDNS failures, pod pending, crashloop, node not ready, upgrade failures, analyze logs, KQL, insights, image pull failures, cold start issues, health probe failures, resource health, root cause of errors, troubleshoot event hubs, troubleshoot service bus, messaging SDK error, AMQP connection failure, message lock lost, lock renewal during long-running processing, service bus dead letter.

OfficialDevOps & Cloud569.8K installs

azure-prepare

microsoft/azure-skills

Prepare azd-based Azure projects for deployment: generates azure.yaml, infrastructure (Bicep/Terraform), and Dockerfiles for the Azure Developer CLI (azd) workflow. USE ONLY when the user explicitly wants to use azd as the deployment tool, or the project already has an azure.yaml file. DO NOT USE FOR: non-azd deployments, Python App Service code-only deploys (use python-appservice-deploy), or cross-cloud migration (use azure-cloud-migrate). WHEN: prepare app for azd, create azure.yaml, set up azd infrastructure, modernize app for Azure with azd, deploy with azd, function app, timer trigger, service bus trigger, event-driven function, managed identity, generate Bicep, generate Terraform, create and deploy to Azure.

OfficialDevOps & Cloud569K installs

azure-ai

microsoft/azure-skills

Use for Azure AI: Search, Speech, OpenAI, Document Intelligence. Helps with search, vector/hybrid search, speech-to-text, text-to-speech, transcription, OCR. WHEN: AI Search, query search, vector search, hybrid search, semantic search, speech-to-text, text-to-speech, transcribe, OCR, convert text to speech.

OfficialDevOps & Cloud568.8K installs

azure-deploy

microsoft/azure-skills

Execute Azure deployments for ALREADY-PREPARED applications that have existing .azure/deployment-plan.md and infrastructure files. DO NOT use this skill when the user asks to CREATE a new application — use azure-prepare instead. This skill runs azd up, azd deploy, terraform apply, and az deployment commands with built-in error recovery. Requires .azure/deployment-plan.md from azure-prepare and validated status from azure-validate. WHEN: "run azd up", "run azd deploy", "execute deployment", "push to production", "push to cloud", "go live", "ship it", "bicep deploy", "terraform apply", "publish to Azure", "launch on Azure". DO NOT USE WHEN: "create and deploy", "build and deploy", "create a new app", "set up infrastructure", "create and deploy to Azure using Terraform" — use azure-prepare for these.

OfficialDevOps & Cloud568.6K installs

azure-validate

microsoft/azure-skills

Pre-deployment validation for Azure readiness. Run deep checks on configuration, infrastructure (Bicep or Terraform), RBAC role assignments, managed identity permissions, and prerequisites before deploying. WHEN: validate my app, check deployment readiness, run preflight checks, verify configuration, check if ready to deploy, validate azure.yaml, validate Bicep, test before deploying, troubleshoot deployment errors, validate Azure Functions, validate function app, validate serverless deployment, verify RBAC roles, check role assignments, review managed identity permissions, what-if analysis, validate Container Apps deployment.

OfficialDevOps & Cloud568K installs

querying-aws-s3

Install

Install

Describe the task

Review the result

Security audits

SKILL.md

Query AWS S3 System Tables

Overview

Decision Tree

Common Tasks

1. Check If Configured

2. Enable (if not configured)

3. Verify Permissions

4. Identify the Target Table

5. Query

Routing: Athena vs Direct API

Troubleshooting

Security Considerations

Least-Privilege IAM Policy

Data Sensitivity

Encryption for Query Results

Audit Trail

Additional Resources

FAQ

More from aws/agent-toolkit-for-aws

Popular devops & cloud skills