What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Kafka can carry PDFs, ZIP archives, and other files as binary record values while the filename travels separately as record metadata, such as a Kafka header. To preserve both, keep the payload byte-oriented through the producer, converters, connectors, and consumer, and verify that each part of the route retains the header. A header by itself does not guarantee end-to-end preservation.
How Kafka carries a file and its name
Kafka records support a value for the payload and headers for associated metadata. That makes a simple arrangement possible: put the file’s bytes in the record value and its filename in a header such as filename. The header name is a convention for your application, not a name mandated by Kafka.
As an Amazon Associate I earn from qualifying purchases.
Kafka’s design describes a standardized binary message format shared by producer, broker, and consumer, so data chunks can pass through that path without modification. That platform-level design does not ensure that an application serializer, Kafka Connect converter, transform, or destination connector will preserve the same bytes and metadata. See the Kafka design documentation and the Kafka header API.
Choose how to represent the payload and filename
| Approach | Payload | Filename | Key consideration |
|---|---|---|---|
| Binary value with header | Original file bytes | Separate record header | Keep the value byte-preserving and confirm that every connector and consumer in the route retains headers. |
| Base64 text representation | File bytes encoded as Base64 text | Header or structured metadata | This is an encoded representation, not the original byte sequence; the receiving boundary must decode it if original bytes are required. |
| Structured envelope | File data and metadata together in an application-defined structure | Included in the envelope | The sources establish no universal advantage over headers; choose a format understood by every participant and test the full route. |
The Kafka Connect 3.6 guide documents a Cast transform that can cast binary data to a string using Base64 encoding. Avoid that conversion when the destination requires the original bytes, or explicitly decode Base64 where the receiving contract calls for it. Consult the Kafka Connect 3.6 user guide alongside the guide for your deployed version.
#1 Best Overall
Keep the filename intact across Kafka Connect
Kafka Connect supports transforms for inserting headers and for copying or moving record fields into headers. Its current user guide describes these header transformations; the Connect header API defines headers as key-value pairs associated with a record. The exact setup depends on your connector and Connect version, so use the documentation matching the deployed system: Kafka Connect user guide and Kafka Connect headers API.
Do not infer that a header survives every route merely because Kafka supports headers. Check each source and sink connector and any intervening transforms for header support and behavior. The Kafka protocol documents the record structure and metadata, but does not certify every third-party connector chain; see the Kafka protocol documentation.
Validate the complete producer-to-consumer path
- Producer: Read the file as bytes and serialize the record value without converting it to text. Attach the filename as metadata, for example in a header named
filename. - Connect converters and transforms: Confirm that the configured converter and every transform preserve the value as binary. Check whether any transform casts binary to a string or otherwise changes its representation.
- Connectors: Verify that the source and destination connectors preserve the record value and carry the filename header through the route. Test their actual versions and configuration rather than assuming uniform behavior.
- Consumer: Use a byte-preserving deserializer for the value and read the corresponding filename header. If the chosen contract uses Base64 text, decode it before treating the result as the original file bytes.
- Round-trip check: Send representative files through the deployed route, then compare the consumed bytes with the input and confirm that the filename is present and unchanged. Include each relevant file type and the largest files the deployment is expected to handle.
Kafka’s message format and configuration options are version-sensitive. Check the documentation corresponding to your broker, Connect runtime, and client versions; the Kafka message-format documentation describes the record format, not a guarantee about every application-level conversion.
Free tools Windows power users keep installed
One-click scans. No signup required.
Account for file size and operational limits
The cited Kafka and Connect documentation does not establish a universal maximum file size or a complete architecture for streaming arbitrarily large files as one record per file. Before choosing that design, check the limits and memory behavior of the deployed broker, producer, connectors, consumer, and retry path against representative files. If a file cannot fit those constraints, do not assume that describing it as “streaming” makes the one-record approach suitable; select an architecture that meets the system’s actual limits.
Quick Recap
Best Value
Rank #4
- Metamorphosis: Franz Kafka (Little Clothbound Classics)
Rank #3
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
