Free tools Windows power users keep installed
One-click scans. No signup required.
A poison message in Kafka is a record that repeatedly fails processing, not a diagnosis. Classify the failure first: use bounded retries for problems likely to clear, and quarantine or send persistent failures to a dead-letter queue (DLQ) with enough context to investigate. Decide whether processing should stop or continue, then make replay safe against duplicate side effects. Kafka Connect, Kafka Streams, and share groups handle these choices differently.
What makes a Kafka message “poison”?
The term describes a symptom: a record keeps failing when a consumer or processing framework tries to handle it. A serialization error, invalid business data, and a temporary downstream outage are different problems, even if each causes processing to fail.
As an Amazon Associate I earn from qualifying purchases.
- Transient failure: A downstream service or infrastructure problem may clear without changing the record. A bounded retry can be appropriate.
- Deterministic record failure: Malformed data or a semantic validation rejection is unlikely to succeed if the same record is tried again unchanged. Repeated retries waste time; preserve the record and its failure context for correction or review.
- Unclear cause: Capture enough diagnostic information to investigate, while avoiding unnecessary exposure of message contents.
Retries, stopping, and continuing all have consequences. A blocking retry can preserve the order in which a partition is processed, but later records may wait behind the failing one. A retry-topic or asynchronous design can let other work proceed, but needs an explicit policy for scheduling and ordering. These are application design trade-offs, not one universal Kafka guarantee.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsChoose whether to retry, stop, or continue
Make the progress-versus-retention decision explicit. Stopping keeps the failed record in the normal processing path but can halt progress. Continuing moves processing forward, so the failed record needs a deliberate route for retention, diagnosis, and eventual reconciliation.
#1 Best Overall
- Retry a likely transient error: Set a finite time budget and delay policy. If the error persists, decide what happens next rather than retrying indefinitely.
- Stop on a failure that must block progress: Use this when processing later records before the failed one would violate application requirements.
- Continue only with a recovery path: Record the failure somewhere operators can inspect, and define how it will be corrected, replayed, or reconciled.
Kafka Connect: configure retries and error handling deliberately
Kafka Connect’s documented default behavior is fail-fast. The Apache Kafka 3.5 Connect User Guide lists default-equivalent settings of errors.retry.timeout=0, errors.log.enable=false, an unset errors.deadletterqueue.topic.name, and errors.tolerance=none. Use the guide for the Connect version actually deployed; these version-specific details should not be assumed to apply unchanged to every release or connector.
The guide’s example uses a ten-minute retry budget and a delay capped at thirty seconds. Those are illustrative configuration values, not universal recommendations:
errors.retry.timeout=600000
errors.retry.delay.max.ms=30000
errors.log.enable=true
errors.log.include.messages=false
errors.deadletterqueue.topic.name=<your-dlq-topic>
errors.tolerance=allIn this example, errors.retry.timeout sets the retry time budget, and errors.retry.delay.max.ms caps the delay. Error-context logging is enabled while message contents are excluded. The DLQ property selects a topic, and errors.tolerance=all permits the connector to continue while reporting errors. Choose a real topic name and verify the complete configuration against the connector and deployed Connect version.
Tolerance is a data-handling decision, not a repair: allowing a connector to continue means failed records may not follow the normal processing path. Kafka Connect warns that logging message contents can expose sensitive information. Also, Connect’s exactly-once support depends on the connector implementation and its ability to use framework capabilities; the example does not promise exactly-once effects in a destination system.
Kafka Streams: fail, continue, or quarantine a bad record
The Kafka 4.2 Streams configuration guide describes deserialization exception handlers that return FAIL or CONTINUE. The built-in log-and-fail handler stops the pipeline on a deserialization failure. The log-and-continue handler logs it and allows later records to be processed.
A custom handler can forward a corrupt record to a quarantine topic. If you choose CONTINUE, do not treat it as successful processing: that record did not complete the ordinary path. Preserve the failure context and decide how it will be corrected or reconciled. Check the configuration guide for the Kafka version you run before relying on these handler details.
Rank #3
Share-group DLQs: what KIP-1191 describes
Kafka’s accepted KIP-1191, “Dead-letter queues for share groups,” was last updated July 16, 2026. It describes DLQ behavior for share groups, where a record repeatedly delivered without successful acknowledgement may be a “poison message.” Under the proposal, a consumer rejection or a reached delivery-attempt limit moves the record through an archiving state and writes to a configured DLQ topic. This is proposal-specific behavior: verify support and configuration in the broker release you deploy rather than assuming every Kafka broker offers it.
Recommended Free Tools
The proposal’s DLQ record can include headers identifying the source topic, partition, offset, group, delivery count, and failure message. Copying the original key, value, and headers is separately configurable and is false by default in the proposal. Including original content may help diagnosis, but can duplicate sensitive data and increase storage or processing costs.
KIP-1191 also describes safeguards: DLQ use must be explicitly enabled, permitted topic names use a configured prefix (the proposal’s default is dlq.), and the broker does not automatically create DLQ topics by default. The proposal warns that writing a DLQ record and updating internal share-group state are not atomic; in rare cases, more than one DLQ record can be written for an undeliverable original. Some DLQ write errors are retried, while other errors are logged and the record proceeds to archived state. Make downstream DLQ handling tolerant of duplicates.
Rank #4
- Metamorphosis: Franz Kafka (Little Clothbound Classics)
A Kafka DLQ preserves failures; it does not fix them
A dead-letter queue is a Kafka topic used to hold records that need a separate handling path. It does not diagnose a cause, repair a record, or guarantee that a replay will be safe. A useful DLQ workflow preserves original identity and relevant failure metadata so an operator can find where the record came from and what happened.
- Keep the source identity, such as topic, partition, and offset, where the mechanism provides it.
- Capture the failure message and delivery context needed to triage the record.
- Decide whether to copy the original key, value, and headers, balancing diagnostic value against privacy and cost.
- Monitor repeated failures and DLQ growth, and assign a process for correction and reconciliation.
Do not assume that putting a record in a DLQ means it has exactly one copy. In particular, KIP-1191 documents the rare duplicate-write case for its proposed share-group behavior.
Replay a Kafka message without repeating unsafe work
The replay shortcut to fix is blindly seeking backward or republishing a DLQ record without coordinating consumer position, output, and external effects. Kafka’s 4.0 message-delivery design documentation explains why duplicates can occur: a consumer can process a record and crash before saving its position, so a replacement consumer receives the record again. Replaying deliberately does not remove that at-least-once delivery risk.
Best Value
- Preserve identity and failure context. Retain enough metadata to identify the original record and understand why it failed.
- Correct or route around the cause. Fix the data, validation rule, or downstream problem before resubmitting work that would fail in the same way.
- Bound the replay. Select the records to retry and control how quickly they re-enter processing; account for ordering requirements.
- Protect side effects. Make destination operations idempotent or deduplicate using a stable record identity. For Kafka-to-Kafka processing, Kafka transactions can atomically commit output records with the input position. An external database or API must cooperate, or the application needs another coordination or idempotency strategy.
- Observe both paths. Watch the original consumer’s lag as well as replay volume and DLQ flow, so recovery work does not silently overwhelm normal processing.
Kafka transactions help coordinate Kafka offsets and Kafka output writes; they are not a general guarantee of exactly-once effects in arbitrary external systems. The exact code change for a seek or republish shortcut depends on the application’s consumer, destination, and side effects.
Keep framework behavior separate
Connect’s error tolerance and DLQ properties, Streams’ exception handlers, and share-group broker behavior are separate mechanisms. Choose the guidance for the component actually processing the record, and validate it against that component’s deployed Kafka version. A configuration or recovery path for one does not automatically configure the others.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
