
aws-messaging-and-streaming
Guides general use of AWS messaging and streaming services. Covers Amazon SQS, Amazon SNS, Amazon EventBridge, Amazon MQ, Amazon Kinesis Data Streams, Amazon Data Firehose, Amazon Managed Service for Apache Flink, and Amazon Managed Streaming for Apache Kafka (MSK). Use when reasoning about messaging and streaming patterns. Also identifies which AWS service owns each customer communication channel: email (Amazon SES), and WhatsApp, SMS, MMS, RCS, voice and mobile push (the AWS End User Messaging family of services). Routes the request to the specialized skill for that channel. Defers to the channel's specialized skill when the user already named a specific channel. In general, use specific skills or documentation searches for detailed service-specific questions. Do NOT use for MSK or Managed Service for Apache Flink questions, prefer specific skills. Does not configure customer communication channels; defers to specific skills.
- Installs
- 8.1K
- Last 24h
- +57
- Trending rank
- #3088
- Security
- Pass
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/aws/agent-toolkit-for-aws --skill aws-messaging-and-streaming- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads aws-messaging-and-streaming automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
Security audits
Independent scans of the skill's instructions and bundled files.
SKILL.md
The instructions your agent reads when the skill is triggered.
AWS Messaging & Streaming Services
When answering AWS messaging and streaming questions, verify specific numbers, versions, limits, and behavioral details from service-specific skills or official AWS documentation. When uncertain, search skills or docs rather than guessing. Fabricated configuration options or incorrect version numbers are worse than admitting uncertainty.
When a question asks about recommended configurations (CloudWatch alarm settings, thresholds, missing data treatment), search for the service-specific skills or documentation rather than relying on general best practices.
Overview
Domain expertise for choosing and using AWS services that move data between producers and consumers. This skill covers two fundamental patterns — messaging and streaming — and the AWS services that implement each. It also marks the boundary with customer communication — messages delivered to people rather than to application components — and routes those questions to the skill or AWS documentation that owns each channel (see Customer Communications (Application-to-Person)). Use this skill to decide which pattern fits a workload, select the right service, and understand how services integrate with each other.
For specific guidance on individual AWS services, see reference files or service-specific Skills.
Streaming and Messaging
What Is Messaging?
Messaging enables decoupled, asynchronous communication between components. A producer sends a message; one or more consumers receive and process it. Once processed, the message is typically deleted. Messaging services handle delivery guarantees, retries, and dead-letter routing.
Key characteristics:
- Messages are consumed once (point-to-point) or fanned out (pub/sub), then removed
- No replay — once acknowledged, a message is gone
- Designed for command/request workloads, task distribution, and event notification
What Is Streaming?
Streaming enables ordered, durable, high-throughput continuous data flow. Producers append records to a log; consumers read from positions in that log. Records persist for a configurable retention period regardless of consumption.
Key characteristics:
- Records are retained and replayable within the retention window
- Strict ordering within a partition/shard
- Multiple independent consumers can read the same data at different positions
- Designed for event sourcing, real-time analytics, change data capture, and continuous processing
Key Differences
| Dimension | Messaging | Streaming |
|---|---|---|
| Data lifecycle | Deleted after consumption | Retained for replay (hours to indefinitely) |
| Ordering | Best-effort (Standard) or per-group (FIFO) | Strict per-partition/shard |
| Consumer model | Competing consumers (work distribution) | Independent readers (fan-out by position) |
| Throughput pattern | Bursty, variable | Sustained, high-volume |
| Replay | Not supported (except DLQ redrive) | Native — seek to any position in retention |
| Typical latency | Milliseconds (push or short-poll) | Milliseconds to low seconds |
| Scaling unit | Concurrency (consumers/pollers) | Partitions or shards |
Messaging Use Cases
- Decoupling microservices with request/response or command patterns
- Distributing work across a pool of competing consumers (task queues)
- Fan-out notifications where each subscriber acts independently
- Workloads that are bursty and benefit from queue buffering
- Migrating existing JMS/AMQP applications (Amazon MQ)
Streaming Use Cases
- Continuous, high-throughput data ingestion (logs, metrics, clickstreams, IoT telemetry)
- Event sourcing where consumers need to replay from any point in time
- Multiple independent consumers processing the same data differently
- Real-time analytics, windowed aggregations, or complex event processing
- Change data capture (CDC) pipelines
Messaging Services
These services are generally used for messaging workloads. Sometimes streaming services (Kinesis Data Streams, Managed Streaming for Apache Kafka) are also used for messaging workloads, depending on exact use case and requirements.
| Service | Best For | Key Differentiator |
|---|---|---|
| Amazon SQS | Task queues, decoupling, buffering | Fully managed, unlimited throughput (Standard), exactly-once (FIFO), fair queues for multi-tenant workloads |
| Amazon SNS | Fan-out, pub/sub notifications | Push to multiple subscribers (SQS, Lambda, HTTP; email/SMS endpoints suit operational alerts — for customer email or SMS see Customer Communications (Application-to-Person)) |
| Amazon EventBridge | Event routing, cross-account/SaaS integration | Content-based filtering, schema registry, 200+ AWS source integrations |
| Amazon MQ | Lift-and-shift of existing JMS/AMQP/MQTT apps | Protocol compatibility (ActiveMQ, RabbitMQ) for legacy migration |
Streaming Services
These services are generally used for streaming workloads.
| Service | Best For | Key Differentiator |
|---|---|---|
| Amazon Kinesis Data Streams | Real-time ingestion with AWS-native consumers | On-demand Advantage mode (instant scaling, no shard management), 1–365 day retention |
| Amazon Data Firehose | Zero-admin delivery to storage/analytics | Auto-scales, buffers, batches, and delivers to destinations |
| Amazon Managed Service for Apache Flink | Complex stream processing (joins, windows, state) | Full Apache Flink runtime — SQL, Java, Python APIs for stateful computation |
| Amazon MSK | Kafka-native workloads, ecosystem compatibility | Apache Kafka API, Express brokers (3x throughput, 20x faster scaling compared to Standard brokers), broad connector ecosystem |
Customer Communications (Application-to-Person)
The services above move data application-to-application, between components of the same application. A separate group of AWS services is application-to-person (A2P): it delivers messages to — or receives them from — recipients outside the application, such as customers and subscribers. The two groups are not interchangeable.
| Channel | Service | Skill |
|---|---|---|
| Amazon SES | amazon-ses | |
| AWS End User Messaging Social | aws-social-messaging | |
| SMS, MMS, RCS, voice | AWS End User Messaging SMS | aws-sms-voice |
| Mobile push | AWS End User Messaging Push | None |
Answer two kinds of question directly from this section: which group a workload belongs to, and which service owns a channel.
For every other customer communication question — setting up a channel, sending through it, or troubleshooting delivery — do not answer from this skill: load the skill named in the table and answer from that.
To load it, use aws___retrieve_skill(skill_name="<skill>") with the exact name from the table when the AWS MCP server is available, or read the skill document from the Agent Toolkit at skills/<skill>/SKILL.md.
Where the table says None, or the named skill cannot be loaded, say so, then answer using the documentation tools (aws___search_documentation, aws___read_documentation) if available, or the AWS documentation for the named service otherwise.
Common Integration Gotchas
-
SQS system vs. user message attributes: Attributes like
AWSTraceHeader(set by X-Ray / EventBridge / Pipes when sending to an SQS DLQ) andSenderId,SentTimestampare SQS system attributes, NOT user message attributes. They are never returned by default fromReceiveMessage— request them explicitly viaAttributeNames=[...](orMessageSystemAttributeNames), separate fromMessageAttributeNameswhich fetches user attributes. This matters for DLQs, where the trace header rides on the system attribute and the user-attributes slot carries the service's failure metadata (e.g. EventBridge'sRULE_ARN,ERROR_CODE). -
SNS → Firehose → S3 record separator: For SNS subscriptions using the
firehoseprotocol that land in S3, records are already newline-delimited by default (NDJSON). Do NOT turn on Firehose'sAppendDelimiterToRecord— SNS emits the newline itself, and enabling the processor produces double newlines. -
EventBridge rule target DLQ + SNS subscription DLQ both need a DLQ queue policy. Attaching the DLQ alone is not enough — the DLQ silently drops messages until its queue policy allows the service principal. EventBridge:
PutTargetswithDeadLetterConfig.Arn=<DLQ>, plus SQS policyAllow sqs:SendMessageforService: events.amazonaws.comwithaws:SourceArn= the rule ARN. SNS:SetSubscriptionAttributesRedrivePolicy={"deadLetterTargetArn":"<DLQ>"}, plus SQS policy allowingService: sns.amazonaws.comscoped by the topic ARN. -
SQS production defaults: long polling + customer-managed encryption. New queues default to short-poll (
ReceiveMessageWaitTimeSeconds=0) and SSE-SQS (AWS-owned key). For production,SetQueueAttributeswithReceiveMessageWaitTimeSeconds=20(long polling) andKmsMasterKeyId=<customer-managed key id/ARN>rather than leavingalias/aws/sqs. -
Broker and Kafka credentials belong in Secrets Manager, not connection strings. Do not hardcode usernames, passwords, or SASL/SCRAM credentials in application config, env vars, JAAS files, or IaC. For Amazon MQ (ActiveMQ/RabbitMQ) store broker users as secrets and fetch at startup; Lambda event source mappings for Amazon MQ require the broker credentials to be supplied as a Secrets Manager secret ARN (
BASIC_AUTH), not inline. For MSK SASL/SCRAM the secret is not optional: it must be named with theAmazonMSK_prefix and encrypted with a customer-managed KMS key (secrets created with the defaultaws/secretsmanagerkey cannot be associated with a cluster), then attached viaBatchAssociateScramSecret. Lambda event source mappings for MSK (SASL/SCRAM or mTLS) and self-managed Kafka also reference a Secrets Manager secret ARN rather than inline credentials. Enable rotation and scope IAM read access (secretsmanager:GetSecretValue) to the consuming role only. See AWS Well-Architected SEC02-BP03 Store and use secrets securely. -
Service-principal resource policies need
aws:SourceArn/aws:SourceAccountconditions. When a queue or topic policy grants a service principal likeevents.amazonaws.com,sns.amazonaws.com, ors3.amazonaws.compermission tosqs:SendMessageorsns:Publish, omitting source conditions opens a confused-deputy hole — any rule, topic, or bucket in any AWS account can drive writes. Scope every such statement withaws:SourceArn(the specific rule/topic/bucket/pipe ARN; useArnLikewith*when the ARN isn't fully known yet) andaws:SourceAccount(your account ID). For S3 event notifications both keys are required because S3 bucket ARNs don't carry the account ID, soaws:SourceArnalone doesn't constrain the account. The same pattern applies to role trust policies for IAM roles used by EventBridge rules and EventBridge Pipes (principalevents.amazonaws.com/pipes.amazonaws.com,aws:SourceArn= the rule or pipe ARN) — not just the DLQ case called out above. See the IAM User Guide on The confused deputy problem.
FAQ
What does the aws-messaging-and-streaming skill do?
Guides general use of AWS messaging and streaming services. Covers Amazon SQS, Amazon SNS, Amazon EventBridge, Amazon MQ, Amazon Kinesis Data Streams, Amazon Data Firehose, Amazon Managed Service for Apache Flink, and Amazon Managed Streaming for Apache Kafka (MSK). Use when reasoning about messaging and streaming patterns. Also identifies which AWS service owns each customer communication channel: email (Amazon SES), and WhatsApp, SMS, MMS, RCS, voice and mobile push (the AWS End User Messaging family of services). Routes the request to the specialized skill for that channel. Defers to the channel's specialized skill when the user already named a specific channel. In general, use specific skills or documentation searches for detailed service-specific questions. Do NOT use for MSK or Managed Service for Apache Flink questions, prefer specific skills. Does not configure customer communication channels; defers to specific skills.
How do I install aws-messaging-and-streaming?
Run npx skills add https://github.com/aws/agent-toolkit-for-aws --skill aws-messaging-and-streaming in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/aws-messaging-and-streaming/ (personal) or .claude/skills/aws-messaging-and-streaming/ (project).
Which AI agents can use aws-messaging-and-streaming?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is aws-messaging-and-streaming safe to install?
Security partners have reviewed it: Gen Agent Trust Hub passed; Socket passed; Snyk passed. As with any skill, read the SKILL.md and bundled scripts before installing.
Who maintains aws-messaging-and-streaming?
It is published by aws in aws/agent-toolkit-for-aws, and it is an official skill from the maker of the technology it covers. It has 8,133 recorded installs.
More from aws/agent-toolkit-for-aws
aws-iam
aws/agent-toolkit-for-aws
Provides verified corrections for IAM behaviors that AI agents frequently get wrong — policy evaluation edge cases, trust policy gotchas, STS session limits, Organizations quirks, and SAML/MFA specifics. Also provides structured workflows for IAM role management and baseline policy generation from application source code or a Terraform plan JSON. Covers condition operator safety (ForAnyValue/ForAllValues with Null checks), bucket policy deny patterns (VPC endpoint restrictions, org paths), confused deputy protection, and service role creation for AWS services (Glue, CloudTrail, Lambda, ECS, etc.) with aws:SourceAccount/aws:SourceArn trust conditions. Applies when creating IAM roles, writing IAM or bucket policies, generating policies from application source code or a Terraform plan JSON, working with STS, Organizations, or condition operators, or any task needing a service or execution role. Does not cover non-IAM authorization like Cognito user-pool policies or app-level RBAC.
aws-serverless
aws/agent-toolkit-for-aws
Routes serverless requests to a compute form factor and the skill that owns it. Applies first to Lambda, API Gateway, EventBridge, Step Functions and event-driven work while the compute, orchestration or integration choice is open — API backends, webhooks, jobs, cron, queue/stream consumers, durable workflows, event routing, Step Functions vs Durable Functions, and Pipes vs native event source mappings for queues and streams. Decides Event Functions on-demand vs Managed Instances, Web Functions, MicroVMs, EventBridge rules/Scheduler/Pipes/bus, HTTP vs REST API. Owns shared Lambda references (cold starts, SnapStart, concurrency, event sources, function URLs, throttling, troubleshooting). Defers to the specialized skill once a request commits to one form factor or service, or describes its defining feature, even unnamed (durable checkpointing and automatic step retries, Firecracker sandbox, managed instances, Node.js server). Not for EC2, ECS/EKS/Fargate or Amplify hosting.
aws-observability
aws/agent-toolkit-for-aws
Builds, configures, debugs, and optimizes AWS observability - operator-symptom questions and detecting Omni vs classic CloudWatch. CloudWatch on an already-reporting service: Log Insights, metric/composite/anomaly alarms, custom metrics/EMF, dashboards, X-Ray/ADOT tracing, canaries, CloudTrail, Dynamic Instrumentation (live breakpoints/snapshots), the Application Signals service map, and fleet health views. CloudWatch Omni on an existing Space: SQL over logs and traces, PromQL over metrics, Omni dashboards, Omni alerts, context graph for root cause, programmatic/IaC access (API/SDK/CLI/CloudFormation), driving Omni from a coding agent or skills, and evaluating AI agent quality from traces - on-demand and online scoring of live traffic, readback, custom evaluators. For first-time setup - creating an Omni Space, granting access, ingestion, or ADOT instrumentation for Application Signals (ServiceEvents, CI/CD metadata) or Omni - use setting-up-cloudwatch-observability. Not for app logging
aws-cdk
aws/agent-toolkit-for-aws
Authors, deploys, and troubleshoots AWS infrastructure using CDK with TypeScript or Python. Covers best practices, stack architecture, and construct patterns. Applies when writing CDK constructs, bootstrapping environments, running cdk deploy/synth/diff, fixing CDK or CloudFormation errors, planning stack structure, importing existing resources, resolving drift, or refactoring stacks without resource replacement.
aws-containers
aws/agent-toolkit-for-aws
Builds and deploys containerized workloads on Elastic Kubernetes Service (EKS), Elastic Container Service (ECS), Fargate, and ECR (Elastic Container Registry). Covers general EKS knowledge, Karpenter, AWS Load Balancer Controller and leveraging various open source Kubernetes projects with EKS. Covers general ECS knowledge, task definitions, Fargate services, ECS Exec, ECS Express Mode and ECS Managed Instances. Covers general Elastic Beanstalk knowledge, Elastic Beanstalk configuration and platforms supported by Elastic Beanstalk. Covers general ECR knowledge, ECR repository setup and lifecycle policies. Includes recommending, enabling, and reading Amazon ECS Action Logs to troubleshoot control-plane failures (deployment rollback/circuit-breaker, task placement, scaling, task replacement). Applies when deploying, debugging, or optimizing containers on AWS. Should be used instead of relying on internal knowledge for these services.
aws-billing-and-cost-management
aws/agent-toolkit-for-aws
Analyze AWS costs, find savings, manage budgets, evaluate Savings Plans and Reserved Instances, right-size EC2/Lambda/RDS/EBS with Compute Optimizer, look up service pricing, query CUR with Athena, detect cost anomalies, scope costs to billing views, and monitor Free Tier usage. Triggers on: AWS bill, cost analysis, reduce spend, savings plan, reserved instance, right-size, budget alert, cost optimization, pricing, free tier, cost anomaly, CUR, cost audit, billing view, billing view ARN.
Popular devops & cloud skills
microsoft-foundry
microsoft/azure-skills
Build, deploy, evaluate, optimize, fine-tune, and manage Microsoft Foundry agents, models, and resources end to end. USE FOR: foundry, azd ai agent, azd provision/deploy, hosted agent scaffold/develop/run/deploy/troubleshoot, prompt agent create, managed harness agent create, GitHub Copilot harness agent, create agent, update agent, add tool to agent, invoke agent, agent insights, evaluate agent, batch eval, continuous eval, continuous monitoring, agent CI/CD, optimize prompt, improve prompt, optimize agent instructions, Agent Optimizer scaffold, dataset curation from traces, deploy model, model fine-tuning (SFT/DPO/RFT), Foundry project, RBAC, role assignment, permissions, quota, capacity, region, deployment failure, AI Services, create Foundry resource, knowledge index, customize deployment, onboard, availability, training-data, grader, distillation, large file upload. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure
azure-diagnostics
microsoft/azure-skills
Debug Azure production issues on Azure using AppLens, Azure Monitor, resource health, and safe triage. WHEN: debug production issues, troubleshoot app service, app service high CPU, app service deployment failure, troubleshoot container apps, container app 503 or target port mismatch, troubleshoot functions, function telemetry missing or stopped after a config/connection-string change, troubleshoot AKS, VM RDP, Linux SSH, VM black screen, can't connect to VM, reset VM password, NSG or firewall blocking, kubectl cannot connect, kube-system/CoreDNS failures, pod pending, crashloop, node not ready, upgrade failures, analyze logs, KQL, insights, image pull failures, cold start issues, health probe failures, resource health, root cause of errors, troubleshoot event hubs, troubleshoot service bus, messaging SDK error, AMQP connection failure, message lock lost, lock renewal during long-running processing, service bus dead letter.
azure-prepare
microsoft/azure-skills
Prepare azd-based Azure projects for deployment: generates azure.yaml, infrastructure (Bicep/Terraform), and Dockerfiles for the Azure Developer CLI (azd) workflow. USE ONLY when the user explicitly wants to use azd as the deployment tool, or the project already has an azure.yaml file. DO NOT USE FOR: non-azd deployments, Python App Service code-only deploys (use python-appservice-deploy), or cross-cloud migration (use azure-cloud-migrate). WHEN: prepare app for azd, create azure.yaml, set up azd infrastructure, modernize app for Azure with azd, deploy with azd, function app, timer trigger, service bus trigger, event-driven function, managed identity, generate Bicep, generate Terraform, create and deploy to Azure.
azure-ai
microsoft/azure-skills
Use for Azure AI: Search, Speech, OpenAI, Document Intelligence. Helps with search, vector/hybrid search, speech-to-text, text-to-speech, transcription, OCR. WHEN: AI Search, query search, vector search, hybrid search, semantic search, speech-to-text, text-to-speech, transcribe, OCR, convert text to speech.
azure-deploy
microsoft/azure-skills
Execute Azure deployments for ALREADY-PREPARED applications that have existing .azure/deployment-plan.md and infrastructure files. DO NOT use this skill when the user asks to CREATE a new application — use azure-prepare instead. This skill runs azd up, azd deploy, terraform apply, and az deployment commands with built-in error recovery. Requires .azure/deployment-plan.md from azure-prepare and validated status from azure-validate. WHEN: "run azd up", "run azd deploy", "execute deployment", "push to production", "push to cloud", "go live", "ship it", "bicep deploy", "terraform apply", "publish to Azure", "launch on Azure". DO NOT USE WHEN: "create and deploy", "build and deploy", "create a new app", "set up infrastructure", "create and deploy to Azure using Terraform" — use azure-prepare for these.
azure-validate
microsoft/azure-skills
Pre-deployment validation for Azure readiness. Run deep checks on configuration, infrastructure (Bicep or Terraform), RBAC role assignments, managed identity permissions, and prerequisites before deploying. WHEN: validate my app, check deployment readiness, run preflight checks, verify configuration, check if ready to deploy, validate azure.yaml, validate Bicep, test before deploying, troubleshoot deployment errors, validate Azure Functions, validate function app, validate serverless deployment, verify RBAC roles, check role assignments, review managed identity permissions, what-if analysis, validate Container Apps deployment.