MCPInstall

kafka-patterns

Apache Kafka 消息队列技能。覆盖生产者 acks=all/1/0 选择决策、幂等性(enable.idempotence + acks=all)配置、消费者手动提交 offset(enable.auto.commit=false)、DLT死信处理(@RetryableTopic/@DltHandler)、@KafkaListener + 异常处理器(DefaultErrorHandler)、顺序消息(同一Key发同一分区)、事务消息。 纠正 LLM:acks=0 丢消息、auto.commit 漏消息、不处理重复消费、不配置 DLT。

Installs
75
Last 24h
—
Trending rank
—
Security
Not audited

Install

Works with every agent the skills CLI supports.

npx skills add https://github.com/full-stack-skills/java-skills --skill kafka-patterns
  1. 01

    Install

    Run the command in your project. The CLI asks which agents to install for.

  2. 02

    Describe the task

    Ask your agent for something this skill covers. It loads kafka-patterns automatically when relevant.

  3. 03

    Review the result

    The agent follows the SKILL.md steps, scripts and references bundled with the skill.

SKILL.md

The instructions your agent reads when the skill is triggered.

Apache Kafka 消息队列

来源:https://kafka.apache.org/documentation/
Spring Kafka:https://docs.spring.io/spring-kafka/reference/

Capability Boundaries

✅ Strong Suits

  1. 生产者配置 — acks(retry/partition)/retries(batchSize/linger)/幂等
  2. 消费者配置 — auto-offset-reset/enable-auto-commit/manual commit
  3. 幂等性保证 — 生产者 enable.idempotence + acks=all + 消费者业务去重
  4. Spring Kafka — @KafkaListener / @RetryableTopic / @DltHandler
  5. DLT死信 — DefaultErrorHandler + DeadLetterPublishingRecoverer
  6. 顺序消息 — 同一Key发送到同一分区
  7. 事务消息 — @Transactional + KafkaTransactionManager

❌ Out of Scope

  1. Kafka Streams → 流处理,不属于消息队列范畴
  2. Kafka Connect → 数据管道,非 LLM 代码问题
  3. Schema Registry → 序列化方案选型

生产者 acks 选择决策

acks值可靠性延迟适用场景
acks=all(-1)⭐⭐⭐ 最高高核心业务(订单/支付/交易) ✅
acks=1⭐⭐ 中中默认值,一般业务
acks=0⭐ 最低低日志/监控/不重要的指标

核心业务必须: acks=all + min.insync.replicas=2(Broker配置) + replication.factor=3

LLM最常犯的错误

#错误正确做法
1acks=0 或 acks=1 丢消息核心业务 acks=all + min.insync.replicas=2
2enable.auto.commit=true(自动提交offset)手动提交,处理完再 acknowledge
3消费者不处理重复消费业务层幂等(唯一键+数据库去重)
4不配置DLT死信,异常消息无限重试DefaultErrorHandler + DeadLetterPublishingRecoverer
5所有消息发到同一个分区用Key路由到分区,同一Key保证顺序
6同步发送 producer.send().get()异步发送 + callback 处理异常
7不配置 enable.idempotence=true幂等生产防止重试导致重复消息
8监听器内异常不处理配置 ErrorHandler,异常→DLT

核心模式

模式 1: 生产者配置(核心业务)

@Bean
public ProducerFactory<String, Object> producerFactory() {
    Map<String, Object> config = new HashMap<>();
    config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    config.put(ProducerConfig.ACKS_CONFIG, "all");                    // ✅ 所有副本确认
    config.put(ProducerConfig.RETRIES_CONFIG, 3);                      // 重试3次
    config.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true);        // ✅ 幂等
    config.put(ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, 5);
    config.put(ProducerConfig.DELIVERY_TIMEOUT_MS_CONFIG, 120_000);    // 投递超时
    config.put(ProducerConfig.COMPRESSION_TYPE_CONFIG, "zstd");        // 压缩
    config.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
    config.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
    return new DefaultKafkaProducerFactory<>(config);
}

模式 2: 消费者手动提交 + DLT

@Bean
public ConsumerFactory<String, Object> consumerFactory() {
    Map<String, Object> config = new HashMap<>();
    config.put(ConsumerConfig.GROUP_ID_CONFIG, "order-group");
    config.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");   // 从最早未消费开始
    config.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false);       // ✅ 手动提交
    config.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, 500);
    config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, JsonDeserializer.class);
    return new DefaultKafkaConsumerFactory<>(config);
}

// ✅ DLT 配置
@Bean
public ConcurrentKafkaListenerContainerFactory<String, Object> kafkaListenerContainerFactory(
        ConsumerFactory<String, Object> cf, KafkaTemplate<String, Object> kt) {
    ConcurrentKafkaListenerContainerFactory<String, Object> factory =
        new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(cf);
    // 重试3次,每次间隔1秒,之后进DLT
    factory.setCommonErrorHandler(new DefaultErrorHandler(
        new DeadLetterPublishingRecoverer(kt),
        new FixedBackOff(1000L, 3L)));
    return factory;
}

模式 3: Spring Kafka 监听器 + DLT

// 方式一:直接 @KafkaListener + DLT ErrorHandler
@KafkaListener(topics = "order-topic", groupId = "order-group")
public void listen(ConsumerRecord<String, OrderEvent> record, Acknowledgment ack) {
    try {
        processOrder(record.value());
        ack.acknowledge();          // ✅ 手动提交 offset
    } catch (Exception e) {
        log.error("处理订单失败: key={}", record.key(), e);
        throw e;                    // 抛出 → ErrorHandler → 重试 → DLT
    }
}

// 方式二:@RetryableTopic 自动重试 + DLT
@RetryableTopic(
    attempts = "4",
    backoff = @Backoff(delay = 1000, multiplier = 2.0, maxDelay = 10000),
    kafkaTemplate = "kafkaTemplate")
@KafkaListener(topics = "order-topic", groupId = "order-group")
public void processOrder(OrderEvent event) {
    // 业务逻辑,失败自动重试(1s→2s→4s→...)
}

@DltHandler
public void handleDlt(OrderEvent event) {
    log.error("消息进入死信队列: {}", event);
    // 人工处理或告警
}

模式 4: 消息发送(异步 + callback)

@Service
public class OrderEventPublisher {
    private final KafkaTemplate<String, Object> kafkaTemplate;

    public void publishOrderCreated(OrderEvent event) {
        String key = String.valueOf(event.getOrderId());  // ✅ 同一订单→同一分区
        ListenableFuture<SendResult<String, Object>> future =
            kafkaTemplate.send("order-topic", key, event);

        // ✅ 异步处理发送结果
        future.addCallback(result -> log.debug("发送成功: {}", result.getRecordMetadata().offset()),
            ex -> log.error("发送失败: key={}", key, ex));
    }
}

模式 5: 业务幂等(消费者端去重)

@KafkaListener(topics = "order-topic")
public void listen(OrderEvent event) {
    // ✅ 业务幂等:用事件ID去重
    String eventId = event.getEventId();
    Boolean existed = redisTemplate.opsForValue().setIfAbsent(
        "kafka:dedup:" + eventId, "1", Duration.ofHours(24));
    if (Boolean.FALSE.equals(existed)) {
        log.debug("重复事件跳过: {}", eventId);
        return;
    }
    // 处理业务
    processOrder(event);
}

Gotchas

  1. acks=all 最安全但最慢 — 日志/监控类消息可用 acks=1
  2. 幂等性 enable.idempotence=true 要求 acks=all + max.in.flight.requests.per.connection<=5
  3. 手动提交 offset 必须 finally 或 Spring Acknowledgment — 漏提交 = 重复消费
  4. DLT 重试次数有限 — 超过重试次数的消息进入 DLT Topic,需要人工处理
  5. 同一 group 内每个分区只被一个消费者消费 — 消费者数 > 分区数 → 多余消费者空闲
  6. offset 从 earliest 开始时重放所有历史消息 — 新消费者组需要注意
  7. 生产者和消费者的序列化器要匹配 — 用 JsonSerializer/JsonDeserializer 替代 String
  8. 异常重试时可能乱序 — 严格顺序场景用单分区或 RabbitMQ
  9. @RetryableTopic 会创建额外 retry topic — 注意 Topic 数量膨胀
  10. 压缩(zstd/snappy) 降低网络带宽 — 推荐 zstd(压缩比高)
  11. max.poll.records 控制每批拉取数量 — 匹配处理速度,避免 rebalance
  12. 大消息(>1MB)需要配置 max.message.bytes — 建议 < 10MB

Data Privacy

本技能不收集、存储或传输任何用户数据。

FAQ

What does the kafka-patterns skill do?

Apache Kafka 消息队列技能。覆盖生产者 acks=all/1/0 选择决策、幂等性(enable.idempotence + acks=all)配置、消费者手动提交 offset(enable.auto.commit=false)、DLT死信处理(@RetryableTopic/@DltHandler)、@KafkaListener + 异常处理器(DefaultErrorHandler)、顺序消息(同一Key发同一分区)、事务消息。 纠正 LLM:acks=0 丢消息、auto.commit 漏消息、不处理重复消费、不配置 DLT。

How do I install kafka-patterns?

Run npx skills add https://github.com/full-stack-skills/java-skills --skill kafka-patterns in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/kafka-patterns/ (personal) or .claude/skills/kafka-patterns/ (project).

Which AI agents can use kafka-patterns?

Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.

Is kafka-patterns safe to install?

No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.

Who maintains kafka-patterns?

It is published by full-stack-skills in full-stack-skills/java-skills. It has 75 recorded installs.

More from full-stack-skills/java-skills

View all →

java-development-manual

full-stack-skills/java-skills

Java开发手册规约集合,基于阿里巴巴Java开发手册(嵩山版)。 涵盖7大维度:编程规约、异常日志、单元测试、安全规约、MySQL数据库、工程结构、设计规约。 当用户需要:(1) 编写或审查Java代码 (2) 检查命名/代码规范 (3) 处理异常和日志 (4) 编写单元测试 (5) 安全编码 (6) 数据库设计 (7) 工程架构设计时使用此skill。 触发词:Java规范、阿里规约、代码规范、开发手册、编程规约、异常处理、单元测试、安全、MySQL、工程结构、设计模式。

Backend & APIs119 installs

mybatis-plus-patterns

full-stack-skills/java-skills

Open the skill to read its SKILL.md instructions.

Backend & APIs114 installs

java-code-comments

full-stack-skills/java-skills

Provides comprehensive guidance for adding Java code comments following industry standards and best practices. This skill helps add class-level comments, method-level comments, and field-level comments to Java code. Use when the user wants to add comments to Java code, needs to document Java classes/methods/fields, wants to improve code documentation, or needs to generate JavaDoc comments. This skill covers Controller, Service, ServiceImpl, Mapper, Model, Entity, BO (Business Object), DTO, VO, and other common Java component types. The skill follows a systematic workflow: scan codebase, identify components, create todo list, and add comments in order (class comments → method comments → field comments).

Backend & APIs112 installs

java-conventions

full-stack-skills/java-skills

统一 Java 项目编码规范与注释规范(SLF4J+Lombok 日志、Bean Lombok 注解选择、判空 Objects/Optional、工具类优先级 Spring→Apache Commons→Hutool→Guava、复杂逻辑用设计模式/简单逻辑不拆分、Controller/Service/ServiceImpl/Mapper 注释一致性)。当需要“按 Java 规范重构/Review/补全注释/统一日志/统一工具类使用/补全 JavaDoc”时使用。

Backend & APIs108 installs

mybatis-patterns

full-stack-skills/java-skills

Open the skill to read its SKILL.md instructions.

Backend & APIs102 installs

redis-redisson-patterns

full-stack-skills/java-skills

Redis + Redisson 缓存与分布式锁技能。覆盖缓存Key命名规范、过期时间设置原则、RedisTemplate序列化选择(JSON vs JDK vs String)、Redisson分布式锁三大模式(watchDog/tryLock 有leaseTime/无leaseTime 决策)、缓存穿透(布隆/空值缓存)/击穿(互斥锁)/雪崩(随机TTL)防护、Caffeine vs Redis选择决策。 纠正 LLM:不设过期时间、StringRedisTemplate和RedisTemplate混用、Redisson锁不 try-finally unlock、看门狗在指定 leaseTime 时不生效。

Data & Databases100 installs

Popular code quality & workflow skills

View category →

grill-me

mattpocock/skills

A relentless interview to sharpen a plan or design.

Scan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.

handoff

mattpocock/skills

Compact the current conversation into a handoff document for another agent to pick up.

domain-modeling

mattpocock/skills

Build and sharpen a project's domain model. Use when discussing codebase terminology, writing or editing a GLOSSARY.md, or recording or editing an ADR.

codebase-design

mattpocock/skills

Shared vocabulary for designing deep modules. Use when the user wants to design or improve a module's interface, find deepening opportunities, decide where a seam goes, make code more testable or AI-navigable, or when another skill needs the deep-module vocabulary.

diagnosing-bugs

mattpocock/skills

Diagnosis loop for hard bugs and performance regressions. Use when the user says "diagnose"/"debug this", or reports something broken/throwing/failing/slow.