s3-explore
Explore and query data on S3, Cloudflare R2, GCS, MinIO, or any S3-compatible storage. Use when the user mentions an s3://, r2://, gs://, or gcs:// URL, asks "what's in this bucket", wants to list remote files, preview remote Parquet/CSV/JSON, or query data on object storage without downloading it. Also triggers when the user wants to know the size, schema, or row count of remote datasets.
- Installs
- 710
- Last 24h
- +8
- Trending rank
- #9403
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/duckdb/duckdb-skills --skill s3-explore- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads s3-explore automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
You are helping the user explore data on remote object storage using DuckDB.
URL: $0
Question: ${1:-list and describe what's there}
Step 1 — Detect provider and set up credentials
Based on the URL or user context, prepend the appropriate secret configuration:
| Provider | URL patterns | Secret setup |
|---|---|---|
| AWS S3 | s3:// | CREATE SECRET (TYPE S3, PROVIDER credential_chain); |
| Cloudflare R2 | r2://, s3:// with R2 endpoint | CREATE SECRET (TYPE R2, PROVIDER credential_chain); |
| GCS | gs://, gcs:// | CREATE SECRET (TYPE GCS, PROVIDER credential_chain); |
| MinIO / custom | s3:// with custom endpoint | CREATE SECRET (TYPE S3, KEY_ID '...', SECRET '...', ENDPOINT '...', USE_SSL true); |
For R2, if the user provides an account ID, the endpoint is <account_id>.r2.cloudflarestorage.com. R2 URLs like r2://bucket/path should be rewritten to s3://bucket/path with the R2 secret.
For public buckets (e.g., Overture Maps, AWS open data), no secret is needed — skip this step.
Always prepend:
LOAD httpfs;
Step 2 — Determine what the URL points to
If the URL looks like a directory or bucket (no file extension, or ends with /), list its contents with sizes:
duckdb -c "
LOAD httpfs;
<SECRET_SETUP>
SELECT filename, (size / 1024 / 1024)::DECIMAL(10,1) AS size_mb, last_modified
FROM read_blob('<URL>/*')
ORDER BY filename
LIMIT 50;
"
Note: only select filename, size, last_modified — never select content, which would download the actual files.
If the URL points to a specific file or glob pattern (has a file extension or contains *), preview it:
duckdb -c "
LOAD httpfs;
<SECRET_SETUP>
DESCRIBE FROM '<URL>';
SELECT count(*) AS row_count FROM '<URL>';
FROM '<URL>' LIMIT 20;
"
For Parquet files, get row counts and sizes from metadata (no data download):
duckdb -c "
LOAD httpfs;
<SECRET_SETUP>
SELECT file_name,
sum(row_group_num_rows) AS total_rows,
(sum(row_group_compressed_bytes) / 1024 / 1024)::DECIMAL(10,1) AS compressed_mb
FROM parquet_metadata('<URL>')
GROUP BY file_name;
"
Step 3 — Answer the question
Using the listing, schema, or sample data, answer:
${1:-list and describe what's there}
If the user asks an analytical question (e.g., "how many rows match X"), write and run the appropriate SQL query. DuckDB pushes predicates down into Parquet on S3, so filtering is efficient even on large remote datasets.
Error handling
duckdb: command not found→ delegate to/duckdb-skills:install-duckdb- Access denied / 403 → suggest the user check credentials:
aws configure, environment variables, or provide explicit key/secret - Bucket not found / 404 → check the URL and region
- Timeout on large listing → suggest narrowing the glob pattern or adding a prefix
FAQ
What does the s3-explore skill do?
Explore and query data on S3, Cloudflare R2, GCS, MinIO, or any S3-compatible storage. Use when the user mentions an s3://, r2://, gs://, or gcs:// URL, asks "what's in this bucket", wants to list remote files, preview remote Parquet/CSV/JSON, or query data on object storage without downloading it. Also triggers when the user wants to know the size, schema, or row count of remote datasets.
How do I install s3-explore?
Run npx skills add https://github.com/duckdb/duckdb-skills --skill s3-explore in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/s3-explore/ (personal) or .claude/skills/s3-explore/ (project).
Which AI agents can use s3-explore?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is s3-explore safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains s3-explore?
It is published by duckdb in duckdb/duckdb-skills. It has 710 recorded installs.
More from duckdb/duckdb-skills
query
duckdb/duckdb-skills
Run SQL queries against the attached DuckDB database or ad-hoc against files. Accepts raw SQL or natural language questions. Uses DuckDB Friendly SQL idioms.
duckdb-docs
duckdb/duckdb-skills
Search DuckDB and DuckLake documentation and blog posts. Returns relevant doc chunks for a question or keyword using full-text search against a locally cached index.
read-file
duckdb/duckdb-skills
Read any data file (CSV, JSON, Parquet, Avro, Excel, spatial, SQLite) or remote URL (S3, HTTPS). Use when user references a data file, asks "what's in this file", or wants to preview/profile a dataset. Not for source code.
attach-db
duckdb/duckdb-skills
Attach a DuckDB database file for use with /duckdb-skills:query. Explores the schema (tables, columns, row counts) and writes a SQL state file so subsequent queries can restore this session automatically via duckdb -init.
install-duckdb
duckdb/duckdb-skills
Install or update DuckDB extensions. Each argument is either a plain extension name (installs from core) or name@repo (e.g. magic@community). Pass --update to update extensions instead of installing.
read-memories
duckdb/duckdb-skills
Search past Claude Code session logs to recall prior decisions, patterns, or unresolved work. Use when user says "do you remember", "what did we do", references past conversations, or you need context from prior sessions.
Popular data & databases skills
supabase-postgres-best-practices
supabase/agent-skills
Postgres best practices maintained by Supabase, for Postgres running anywhere. Load this skill BEFORE writing or changing anything that lives in a Postgres database: creating or altering tables and columns (including choosing column types), schema design, migrations and declarative schema files, RLS policies and the tests that verify them, indexes, triggers, database functions, queues and scheduled jobs (pgcron, pgmq), vector/semantic search (pgvector), and restoring dumps (pgrestore) or importing data. Also load it when diagnosing slow queries, high CPU, timeouts, EXPLAIN plans, connection exhaustion, locking, bloat, or rows visible to the wrong user or tenant. This is not just a performance guide — schema, migration, security, and SQL authoring tasks need these rules too, even for a one-column change or a single query.
prisma-client-api
prisma/skills
Prisma Client API reference covering model queries, filters, operators, and client methods. Use when writing database queries, using CRUD operations, filtering data, or configuring Prisma Client. Triggers on "prisma query", "findMany", "create", "update", "delete", "$transaction".
prisma-database-setup
prisma/skills
Deprecated compatibility name for prisma-orm-setup. Use only when a user or existing integration explicitly invokes prisma-database-setup; otherwise select prisma-orm-setup.
prisma-cli
prisma/skills
Prisma ORM 7 CLI reference for existing Prisma 7 applications or explicitly selected Prisma 7 workflows. Covers prisma init, generate, migrate, db, dev, studio, and mcp. For setup, connection repair, or Prisma 8, use prisma-orm-setup; not the Prisma Platform CLI.
prisma-postgres
prisma/skills
Deprecated compatibility name for prisma-postgres-setup. Use only when a user or existing integration explicitly invokes prisma-postgres; otherwise select prisma-postgres-setup.
prisma-driver-adapter-implementation
prisma/skills
Required reference for Prisma ORM 7 SQL driver adapter work. Use when implementing or modifying adapters, adding database drivers, or touching SqlDriverAdapter, Transaction, savepoint, result mapping, or DriverAdapterError behavior. Covers current transaction lifecycle, optional savepoint hooks, original database-error preservation, and verification.