Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →You can send payloads larger than the standard SQS and SNS message limit from Kotlin without adding AWS’s Java extended-client libraries. The pattern is simple: store the payload in Amazon S3 and send a small reference through SQS or SNS. The part the Java libraries normally handle, including upload, reference format, retrieval, and cleanup, then becomes code your team owns and must keep compatible with every producer and consumer.
AWS documents the extended-client approach only as Java libraries. Kotlin runs on the JVM and can call them, but AWS does not publish a Kotlin-native version. “Without Java baggage” therefore means avoiding the library and its AWS SDK for Java dependency, not avoiding the JVM. The sections below separate what AWS documents from what you would need to design yourself.
As an Amazon Associate I earn from qualifying purchases.
How the offloading pattern works
SQS and SNS messages have a standard size limit of 256 KB, as stated in AWS’s guide to managing large Amazon SQS messages using Java and Amazon S3. The pattern works around that limit in four steps:
Free tools Windows power users keep installed
One-click scans. No signup required.
- The publisher writes the full payload to an S3 object.
- The publisher sends a message to SQS or SNS that contains only a reference to that object.
- The consumer reads the reference, fetches the object from S3, and processes the payload.
- Someone, or some rule, deletes the object once it is no longer needed.
The message service carries only the pointer. Nothing in the messaging layer changes the payload, which is why every consumer must know how to resolve the pointer.
#1 Best Overall
What AWS documents and ships
The SQS extended client
AWS’s SQS guide says the Amazon SQS Extended Client Library for Java stores a message payload in an S3 bucket and puts a reference to the object in SQS. The guide describes its use for payloads from 256 KB up to 2 GB. It also states the scope limit directly: you can use the library to manage SQS messages through Amazon S3 “only with the AWS SDK for Java.” The library source is published in the AWS Labs Amazon SQS Java Extended Client Library repository.
The SNS extended client
The SNS side is documented in the Amazon SNS Extended Client Library for Java guide. The publisher uses the SNS extended client to store the payload in S3 and publish the reference. The guide describes configuring a size threshold, an always-through-S3 mode, the bucket, and a custom AWS KMS key. The AWS Labs SNS Java repository states the same 2 GB maximum.
Rank #2
A worked example is available in AWS’s SDK example for publishing a large message to Amazon SNS with Amazon S3. It shows an SNS topic with an SQS subscriber, where the subscriber’s SQS extended client retrieves the content. That example also configures raw message delivery, so the consumer receives the pointer without SNS’s notification wrapper.
AWS also publishes a Python extended-client guide for SNS. It does not change the Kotlin question, but it confirms that the Java-only scope applies to the libraries this article discusses, not to the pattern itself.
Limits in practice
The 256 KB figure is the standard message-size limit. The 2 GB figure is the payload range AWS describes for its Java extended clients. Treat it as library capability, not as a guarantee of latency, throughput, or service behavior in every configuration.
Can Kotlin call the Java libraries?
Kotlin compiles to JVM bytecode and can call Java classes, so the extended clients are technically usable from a Kotlin module. The AWS guide’s scope statement still applies, though: the library is meant for use with the AWS SDK for Java. Calling it from Kotlin therefore brings the library and the SDK it depends on into your build. That is the cost the phrase “Java baggage” refers to.
Calling the library is the right choice when you need interoperability with existing Java producers or consumers and want AWS’s own reference handling. Building your own adapter makes sense when you want a Kotlin-first API, a smaller dependency surface, or full control over the pointer format and failure policy. The trade-off is that you then own everything the library would otherwise do.
Building a Kotlin-owned S3 offload adapter
AWS does not specify a Kotlin API for this pattern, and the sources do not describe a tested Kotlin implementation. What follows is an engineering design built on the documented mechanism. Each step below is your responsibility.
Best Value
Publisher path
- Serialize the payload to bytes. Choose a format and version it, because consumers will depend on it.
- Compare the size with your threshold. Below the threshold, send the body directly. Above it, or always if you choose always-through-S3, continue.
- Upload the bytes to a bucket you select, under a key you generate, such as a prefix plus a UUID. Use a bucket in the same region as your queue or topic unless you have a reason not to.
- Publish a pointer envelope to SQS or SNS. Only send the message after the upload succeeds.
Consumer path
- Read the message and detect whether the body is a pointer envelope or a direct payload.
- Fetch the S3 object using the bucket and key from the envelope.
- Verify the object before processing, for example by checking a size or checksum field you included in the envelope.
- Process the payload, then acknowledge the message. Do not delete the S3 object before acknowledgement, because a retry would then fail.
Cleanup path
- Set an explicit retention period for offloaded objects that is longer than your maximum message retention and redelivery window.
- Apply it with an S3 lifecycle rule, or delete the object in the consumer after successful processing if your design requires immediate removal.
- Log orphaned objects, meaning uploads that never got a matching message, and clean them up on a schedule.
The pointer contract
The envelope is the interface between every publisher and consumer, so define it once and version it. A minimal example looks like this:
{
"v": 1,
"type": "s3-pointer",
"bucket": "example-payload-bucket",
"key": "orders/2026/10/9/3f2a7c.json",
"size": 1048576,
"sha256": "9c1f...e0a4"
}This is one contract your team defines. It is not the format the AWS libraries write. If producers or consumers already use the Java extended client, read the library source to confirm its reference format before you choose your own, or the two sides will not interoperate. A consumer that expects a full body and receives only this envelope will fail, so route messages by a type field rather than guessing from content.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Choosing between the AWS Java client and a Kotlin adapter
| Factor | AWS Java extended client | Kotlin-owned adapter |
|---|---|---|
| Documentation from AWS | Yes, for SQS and SNS, as Java libraries | None published for Kotlin; you define the contract |
| Dependencies | Requires the AWS SDK for Java | Your choice of S3 and messaging client |
| Upload and retrieval | Handled by the library | Your code |
| Pointer format | Set by the library | Set by your envelope definition |
| Interoperability with existing Java producers and consumers | Native to the library | Only if you match the library’s reference format |
| Retries, partial failures, cleanup | Library behavior, per the AWS guides | Your responsibility, including retention policy |
| Documented payload range | Up to 2 GB for the Java libraries | Bounded by your design and S3 object limits |
The comparison axes are engineering judgment. AWS’s documentation covers the Java library’s behavior; it does not benchmark either option.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsOperational choices to settle first
- Threshold or always-through-S3. A threshold keeps small messages inline and reduces S3 calls. Always-through-S3 gives consumers one code path. AWS documents both modes for the Java libraries.
- Bucket and region. Keep the bucket in the same region as your queue or topic, and scope access per producer and consumer.
- Encryption. If you need a customer-managed key, plan for a custom AWS KMS key. AWS’s SNS guide documents KMS key configuration for the Java client, and your adapter must grant the same permissions to producers and consumers.
- IAM permissions. Producers need write access to the offload prefix, consumers need read access, and only the cleanup role should delete objects.
- Retries and partial failures. An upload that succeeds followed by a failed publish leaves an orphaned object. A publish that succeeds before the upload finishes is worse, because a consumer will look for an object that does not exist yet. Upload first, publish second.
- Retention. Object lifetime must exceed the longest time a message can remain in a queue or be redelivered.
SNS to SQS fan-out
When one SNS topic feeds several SQS queues, each subscriber receives the same pointer. Each consumer must resolve it independently, which means each needs read access to the bucket. Raw message delivery matters here: without it, SNS wraps the message in its own JSON notification, and consumers must unwrap that before reading the pointer. The AWS example’s raw-delivery setting is what makes the pointer appear directly in the SQS body.
Troubleshooting common failures
- Consumer receives a pointer where it expected a body. The consumer is not checking the envelope type, or a producer changed the contract without a version bump. Route on the type field and version the envelope.
- Access denied when fetching the object. The consumer role lacks
s3:GetObjecton the offload prefix, or the object uses a KMS key the consumer cannot decrypt with. - Object not found on retry. Cleanup ran too early, or the consumer deleted the object before acknowledging the message. Move deletion behind the retention window.
- Growing storage with no matching traffic. Uploads succeed while publishes fail. Add a reconciliation job that compares recent keys with delivered envelopes.
Figures and what they do not tell you
- 256 KB: the standard SQS and SNS message-size limit, as described in AWS’s SQS extended-client guide.
- Up to 2 GB: the payload range AWS documents for its Java extended clients. This is a library capability, not a performance promise.
- 2020: the year AWS announced SNS client support for payloads up to 2 GB, as recorded in the AWS announcement from August 2020. It is historical context.
The sources do not establish any adoption, latency, throughput, reliability, or cost figure for this pattern. Offloading is not automatically cheaper, because every large message adds an S3 write, a read, and storage that someone must manage.
Quick Recap
“
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
