storing-and-querying-vectors
Store and query vector embeddings using Amazon S3 Vectors, a cost-effective long-term vector storage service with its own API namespace (s3vectors). Triggers on: create S3 vector bucket, vector index, store embeddings, semantic search, RAG vector storage, similarity search, vector database, migrate from other vector databases. Do NOT use for: querying tabular data (use querying-data-lake), S3 object storage, or hundreds/thousands of sustained QPS (use OpenSearch).
- Installs
- 6.6K
- Last 24h
- +40
- Trending rank
- #3914
- Security
- Pass
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/aws/agent-toolkit-for-aws --skill storing-and-querying-vectors- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads storing-and-querying-vectors automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
Security audits
Independent scans of the skill's instructions and bundled files.
SKILL.md
The instructions your agent reads when the skill is triggered.
Store and Query Vectors with Amazon S3 Vectors
Overview
Amazon S3 Vectors is a cost-effective AWS service for storing and querying vector embeddings at scale. Optimized for long-term storage with subsecond latency for cold queries, as low as 100ms for warm queries.
Decision Guide
- Hundreds/thousands of sustained queries per second (QPS): Wrong tool. Recommend OpenSearch.
- Hybrid search, aggregations, faceted search: Recommend OpenSearch with S3 Vectors as storage engine. For OpenSearch integration, search AWS docs for
"Using S3 Vectors with OpenSearch Service". - Tiered (bulk + hot): S3 Vectors for storage + OpenSearch Serverless for real-time. See
references/limits-and-patterns.md. - Cost-effective storage, infrequent queries, RAG: S3 Vectors is the right fit. Proceed.
For latest guidance, search AWS docs for "S3 Vectors best practices".
Common Tasks
Classify the request before starting:
- Simple query: Existing index, skip to Step 6
- Standard: You MUST list existing indexes first and suggest reusing if relevant. Else, new index + store vectors, follow Steps 2-6
- Migration or multi-tenant: Read
references/limits-and-patterns.mdfirst, then Steps 2-6
You MUST execute commands using AWS MCP server tools when connected. Fall back to AWS CLI only if AWS MCP is unavailable. You MUST explain each step to the user before executing.
1. Verify Dependencies
Constraints:
- You MUST check whether AWS MCP tools or AWS CLI is available and inform user if missing
- You MUST confirm target AWS region
2. Create a Vector Bucket
You MUST confirm bucket name with user. Names: 3-63 chars, lowercase letters, numbers, hyphens only. Encryption (SSE-S3 default or SSE-KMS for compliance) is immutable after creation.
aws s3vectors create-vector-bucket \
--vector-bucket-name <BUCKET_NAME>
Constraints:
- You MUST explain encryption cannot be changed after creation
- For SSE-KMS, KMS key policy MUST grant
kms:GenerateDataKeyandkms:Decryptto the S3 Vectors service principalindexing.s3vectors.amazonaws.com. You MUST use full KMS key ARN (not alias). Seereferences/limits-and-patterns.mdfor command example.
3. Create a Vector Index
Every parameter is immutable after creation.
Pre-flight checklist (confirm ALL with user):
- Dimension (required, integer 1-4096) -- MUST match embedding model output
- Distance metric (required) --
cosineoreuclidean. Use embedding model's recommended metric; - Non-filterable metadata keys (optional, max 10, 1-63 chars) -- Declare at creation or lose forever. For Bedrock Knowledge Bases integration, search AWS docs for
"S3 Vectors Bedrock Knowledge Bases prerequisites"to get the required key names. - Encryption (optional) -- Inherits from bucket. Override per-index if needed.
aws s3vectors create-index \
--vector-bucket-name <BUCKET_NAME> \
--index-name <INDEX_NAME> \
--dimension <DIM> \
--distance-metric <cosine|euclidean> \
--data-type float32 \
--metadata-configuration '{"nonFilterableMetadataKeys":["<KEY1>","<KEY2>"]}'
Omit --metadata-configuration if no non-filterable keys are needed.
Index names: 3-63 chars, lowercase, numbers, hyphens, dots. Unique within bucket. Filterable metadata: 2 KB limit. Total metadata (filterable + non-filterable combined): 40 KB. See references/metadata-filtering.md.
4. Generate Embeddings (if needed)
Skip to Step 5 (store) or Step 6 (query) if user already has embeddings.
Constraints:
- You MUST ask which embedding model to use if not specified
- You MUST NOT assume a default model
- Dimension MUST match Step 3
- You MUST use the same model for both storing and querying
Generate embeddings with Bedrock invoke-model:
aws bedrock-runtime invoke-model \
--model-id <MODEL_ID> \
--content-type application/json \
--cli-binary-format raw-in-base64-out \
--body '{"inputText": "your text"}' \
invoke-model-output.json
You MUST use --cli-binary-format raw-in-base64-out for CLI v2. Output file is required for CLI. The response key is model-dependent (e.g., embedding for Titan, embeddings for Cohere). For Titan, parse with json.load(open('invoke-model-output.json'))['embedding']. Use embedding array as float32 in put-vectors or query-vectors. For batch embedding generation, use AWS SDK or CLI.
5. Put Vectors
aws s3vectors put-vectors \
--vector-bucket-name <BUCKET_NAME> \
--index-name <INDEX_NAME> \
--vectors '[{"key":"<ID>","data":{"float32":[<EMBEDDING>]},"metadata":{"topic":"science"}}]'
Constraints:
- You MUST NOT exceed 500 vectors per call
- You SHOULD batch vectors for cost optimization
- For bulk operations, You SHOULD use an SDK instead of CLI -- vector payloads may be too large for shell arguments
- You MUST implement retry with backoff on
429 TooManyRequestsException - See
references/limits-and-patterns.mdfor batch patterns
6. Query Vectors
Generate embedding if needed (Step 4), then query:
aws s3vectors query-vectors \
--vector-bucket-name <BUCKET_NAME> \
--index-name <INDEX_NAME> \
--query-vector '{"float32":[<EMBEDDING>]}' \
--top-k 10 \
--return-distance
Optional: add --return-metadata and/or --filter '{"topic":{"$eq":"science"}}' (both require GetVectors permission). See references/metadata-filtering.md.
Example response body: {"vectors": [{"key": "id1", "distance": 0.45, "metadata": {"topic": "science"}}, ...], "distanceMetric": "cosine"}
Constraints:
- Using
--filteror--return-metadatarequires boths3vectors:QueryVectorsANDs3vectors:GetVectorsIAM permissions. Without GetVectors, these options return 403.
Troubleshooting
| Error | Cause | Fix |
|---|---|---|
DimensionMismatch | Dims don't match index | Use matching model, or delete/recreate index (confirm with user -- destroys all vectors). |
403 Forbidden with --filter or --return-metadata | Missing s3vectors:GetVectors | Add s3vectors:GetVectors to IAM policy. |
Fewer results than --top-k | Few vectors match filter | Expected -- filtering is inline. Broaden filter. |
429 TooManyRequestsException | Exceeded per-index rate limits | Retry with backoff. Shard across indexes for sustained throughput. Search AWS docs for "S3 Vectors limitations and restrictions" for current limits. |
AccessDeniedException | Missing s3vectors:* IAM actions | S3 Vectors uses s3vectors:* namespace, not s3:*. Update IAM policy. |
RequestTimeoutException or service unavailable | Request timeout or region not supported | Retry request. For regional availability, search AWS docs for "S3 Vectors limitations and restrictions". |
Additional Resources
- limits-and-patterns.md -- Multi-tenant patterns, batch ingestion, SSE-KMS, migration
- metadata-filtering.md -- Filter operators, non-filterable metadata, Bedrock KB keys
FAQ
What does the storing-and-querying-vectors skill do?
Store and query vector embeddings using Amazon S3 Vectors, a cost-effective long-term vector storage service with its own API namespace (s3vectors). Triggers on: create S3 vector bucket, vector index, store embeddings, semantic search, RAG vector storage, similarity search, vector database, migrate from other vector databases. Do NOT use for: querying tabular data (use querying-data-lake), S3 object storage, or hundreds/thousands of sustained QPS (use OpenSearch).
How do I install storing-and-querying-vectors?
Run npx skills add https://github.com/aws/agent-toolkit-for-aws --skill storing-and-querying-vectors in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/storing-and-querying-vectors/ (personal) or .claude/skills/storing-and-querying-vectors/ (project).
Which AI agents can use storing-and-querying-vectors?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is storing-and-querying-vectors safe to install?
Security partners have reviewed it: Gen Agent Trust Hub passed; Socket passed; Snyk passed. As with any skill, read the SKILL.md and bundled scripts before installing.
Who maintains storing-and-querying-vectors?
It is published by aws in aws/agent-toolkit-for-aws, and it is an official skill from the maker of the technology it covers. It has 6,645 recorded installs.
More from aws/agent-toolkit-for-aws
aws-iam
aws/agent-toolkit-for-aws
Provides verified corrections for IAM behaviors that AI agents frequently get wrong — policy evaluation edge cases, trust policy gotchas, STS session limits, Organizations quirks, and SAML/MFA specifics. Also provides structured workflows for IAM role management and baseline policy generation from application source code or a Terraform plan JSON. Covers condition operator safety (ForAnyValue/ForAllValues with Null checks), bucket policy deny patterns (VPC endpoint restrictions, org paths), confused deputy protection, and service role creation for AWS services (Glue, CloudTrail, Lambda, ECS, etc.) with aws:SourceAccount/aws:SourceArn trust conditions. Applies when creating IAM roles, writing IAM or bucket policies, generating policies from application source code or a Terraform plan JSON, working with STS, Organizations, or condition operators, or any task needing a service or execution role. Does not cover non-IAM authorization like Cognito user-pool policies or app-level RBAC.
aws-serverless
aws/agent-toolkit-for-aws
Routes serverless requests to a compute form factor and the skill that owns it. Applies first to Lambda, API Gateway, EventBridge, Step Functions and event-driven work while the compute, orchestration or integration choice is open — API backends, webhooks, jobs, cron, queue/stream consumers, durable workflows, event routing, Step Functions vs Durable Functions, and Pipes vs native event source mappings for queues and streams. Decides Event Functions on-demand vs Managed Instances, Web Functions, MicroVMs, EventBridge rules/Scheduler/Pipes/bus, HTTP vs REST API. Owns shared Lambda references (cold starts, SnapStart, concurrency, event sources, function URLs, throttling, troubleshooting). Defers to the specialized skill once a request commits to one form factor or service, or describes its defining feature, even unnamed (durable checkpointing and automatic step retries, Firecracker sandbox, managed instances, Node.js server). Not for EC2, ECS/EKS/Fargate or Amplify hosting.
aws-observability
aws/agent-toolkit-for-aws
Builds, configures, debugs, and optimizes AWS observability - operator-symptom questions and detecting Omni vs classic CloudWatch. CloudWatch on an already-reporting service: Log Insights, metric/composite/anomaly alarms, custom metrics/EMF, dashboards, X-Ray/ADOT tracing, canaries, CloudTrail, Dynamic Instrumentation (live breakpoints/snapshots), the Application Signals service map, and fleet health views. CloudWatch Omni on an existing Space: SQL over logs and traces, PromQL over metrics, Omni dashboards, Omni alerts, context graph for root cause, programmatic/IaC access (API/SDK/CLI/CloudFormation), driving Omni from a coding agent or skills, and evaluating AI agent quality from traces - on-demand and online scoring of live traffic, readback, custom evaluators. For first-time setup - creating an Omni Space, granting access, ingestion, or ADOT instrumentation for Application Signals (ServiceEvents, CI/CD metadata) or Omni - use setting-up-cloudwatch-observability. Not for app logging
aws-cdk
aws/agent-toolkit-for-aws
Authors, deploys, and troubleshoots AWS infrastructure using CDK with TypeScript or Python. Covers best practices, stack architecture, and construct patterns. Applies when writing CDK constructs, bootstrapping environments, running cdk deploy/synth/diff, fixing CDK or CloudFormation errors, planning stack structure, importing existing resources, resolving drift, or refactoring stacks without resource replacement.
aws-containers
aws/agent-toolkit-for-aws
Builds and deploys containerized workloads on Elastic Kubernetes Service (EKS), Elastic Container Service (ECS), Fargate, and ECR (Elastic Container Registry). Covers general EKS knowledge, Karpenter, AWS Load Balancer Controller and leveraging various open source Kubernetes projects with EKS. Covers general ECS knowledge, task definitions, Fargate services, ECS Exec, ECS Express Mode and ECS Managed Instances. Covers general Elastic Beanstalk knowledge, Elastic Beanstalk configuration and platforms supported by Elastic Beanstalk. Covers general ECR knowledge, ECR repository setup and lifecycle policies. Includes recommending, enabling, and reading Amazon ECS Action Logs to troubleshoot control-plane failures (deployment rollback/circuit-breaker, task placement, scaling, task replacement). Applies when deploying, debugging, or optimizing containers on AWS. Should be used instead of relying on internal knowledge for these services.
aws-billing-and-cost-management
aws/agent-toolkit-for-aws
Analyze AWS costs, find savings, manage budgets, evaluate Savings Plans and Reserved Instances, right-size EC2/Lambda/RDS/EBS with Compute Optimizer, look up service pricing, query CUR with Athena, detect cost anomalies, scope costs to billing views, and monitor Free Tier usage. Triggers on: AWS bill, cost analysis, reduce spend, savings plan, reserved instance, right-size, budget alert, cost optimization, pricing, free tier, cost anomaly, CUR, cost audit, billing view, billing view ARN.
Popular ai & agents skills
google-agents-cli-publish
google/agents-cli
This skill should be used when the user wants to "publish an agent", "publish my ADK agent", "register an agent with Gemini", "publish to Gemini" (formerly Gemini Enterprise), or needs guidance on the agents-cli publish gemini command. Also use when the user wants to "manage agents in Agent Registry", "list/update/delete registered agents", or "register an MCP server". Covers ADK vs A2A registration modes, programmatic and interactive usage, flag reference, auto-detection from deployment metadata, Agent Registry fleet management, and troubleshooting. Part of the agents-cli skills suite. Do NOT use for deployment (use google-agents-cli-deploy).
google-agents-cli-eval
google/agents-cli
This skill should be used when the user wants to "run an evaluation", "evaluate my agent", "evaluate my ADK agent", "write an eval dataset", "analyze eval failures", "compare eval results", "optimize agent", or needs guidance on the Gemini platform eval methodology and the Quality Flywheel. Covers eval metrics, dataset schema, LLM-as-judge scoring, and common failure causes. Applies to any agents-cli project, whatever framework the agent is written in. Do NOT use for agent API code patterns (ADK: use google-agents-cli-adk-code), deployment (use google-agents-cli-deploy), or project scaffolding (use google-agents-cli-scaffold).
skill-creator
anthropics/skills
Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.
codex-pet
prime-skills/runcomfy-agent-skills
Codex Pet generator on RunComfy. Build a Codex-compatible Codex Pet spritesheet.webp + pet.json from a single reference image, drop it into ${CODEXHOME:-$HOME/.codex}/pets/ / and Codex picks it up as a custom Codex Pet next to the 8 built-ins. This skill produces the exact Codex Pet atlas Codex expects (1536x1872 PNG/WebP, 8 cols x 9 rows, 192x208 cells, 9 animation states — idle, running-right, running-left, waving, jumping, failed, waiting, running, review). Calls OpenAI GPT Image 2 edit ONCE via the local RunComfy CLI as runcomfy run openai/gpt-image-2/edit to produce a canonical Codex Pet pose, then assembles all 9 animation rows programmatically with ImageMagick micro-transforms — no Codex Pro, no $imagegen, no OPENAIAPIKEY required, only RUNCOMFYTOKEN. Triggers on "codex pet", "create codex pet", "make codex pet", "hatch codex pet", "/hatch image", "desktop pet codex", "codex pets", "spritesheet.webp", or any explicit ask to build a custom pet for OpenAI Codex.
full-output-enforcement
leonxlnx/taste-skill
Overrides default LLM truncation behavior. Enforces complete code generation, bans placeholder patterns, and handles token-limit splits cleanly. Apply to any task requiring exhaustive, unabridged output.
executing-plans
obra/superpowers
Use when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available