To publish a DynamoDB change reliably, write the business item and an outbox event together in one TransactWriteItems operation, then relay the outbox record from DynamoDB Streams to SQS. Make the consumer safe to run more than once: retries and duplicate deliveries are expected, and console settings alone cannot make application side effects idempotent.
Why use an outbox between DynamoDB and SQS?
A business write and an SQS send are separate operations. If the database write succeeds but sending fails, downstream systems may never hear about the change. If the message is sent but the database write fails, consumers may act on a change that never committed.
As an Amazon Associate I earn from qualifying purchases.
The outbox pattern closes that gap at the point where the business data is written: store an event record in DynamoDB in the same transaction as the business change. A relay later reads the committed outbox record from the table’s stream and publishes it to SQS. The relay can retry without inventing an event for a failed business write.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →This does not make the entire path exactly once. The relay can publish more than once, and SQS Standard provides at-least-once delivery. The design must therefore handle retries at both the publishing and consumption stages.
#1 Best Overall
What should the outbox event contain?
Choose the event contract before configuring the services. Give each event a stable identifier that remains unchanged across relay retries, and include enough information for consumers to validate and process it.
- Event ID: a stable unique identifier used by consumers for deduplication.
- Event type and schema version: identify what happened and how to interpret the payload.
- Entity key: identify the business item or aggregate the event concerns.
- Creation time: record when the application created the event.
- Payload: include the fields consumers need, rather than assuming the relay or consumer can reconstruct historical state later.
For example, an order update might write the changed order and an outbox item containing an event ID, an event type such as OrderUpdated, a schema version, the order key, a creation time, and the relevant order data. This is an illustrative contract, not a required AWS format.
Decide what happens to outbox records after publication. You can retain them for audit or replay, or mark or expire them under an explicit retention policy. Do not remove an event until there is a dependable recovery path for a failed or uncertain publish.
How do you write the change and event atomically?
Have the application issue a single DynamoDB TransactWriteItems request containing both the business-item write and the outbox-item write. DynamoDB transactions are all-or-nothing within the Region of the write: both writes commit, or neither does.
TransactWriteItems also accepts an optional client token to make retries of the same request idempotent for a limited period. AWS documents that token window as 10 minutes after the request finishes. It helps with retries of that transaction request; it is not durable consumer deduplication and does not replace a stable event ID.
Keep the transaction in the application write path. The DynamoDB console can configure the table and stream, but it does not create the application code that constructs the event or performs the transactional write.
How do you enable a DynamoDB Stream in the console?
- In the DynamoDB console, open Tables and select the table that stores both the business items and outbox items.
- Open Exports and streams, turn on DynamoDB Streams, and choose a stream view type that exposes the event fields your relay needs.
- Save the stream ARN for the Lambda trigger you will configure next.
The view type determines which item data a stream record exposes. Choose based on your event representation and relay needs:
Free tools Windows power users keep installed
One-click scans. No signup required.
| View type | What it exposes | When it may fit |
|---|---|---|
KEYS_ONLY |
Key attributes only | When the relay can look up the needed data separately; that adds a read and may not provide a historical snapshot. |
NEW_IMAGE |
The item’s new image | Often useful when an inserted outbox item contains the complete event envelope. |
OLD_IMAGE |
The item’s old image | When the relay needs the prior item state. |
NEW_AND_OLD_IMAGES |
Both the new and old images | When the event needs to describe a transition between states. |
These view types are not interchangeable after the stream is created: the view type cannot be edited in place. Changing it requires disabling and recreating the stream, so choose deliberately. AWS’s DynamoDB Streams and Lambda tutorial demonstrates enabling a stream in the console, attaching a Lambda event source, and checking changes to table items.
How do you relay outbox records from the stream to SQS?
- Create the SQS destination queue and a Lambda function that will act as the relay.
- Give the relay’s execution role permission to read the relevant DynamoDB stream and send messages to the specific SQS queue.
- Add a DynamoDB Streams trigger to the relay Lambda using the stream ARN saved from the table settings.
- In the relay code, select only the intended outbox records, shape each SQS message from the stored event envelope, and preserve the event ID unchanged.
DynamoDB Streams is a change-capture mechanism, not a domain-event classifier. Your relay must distinguish publishable outbox writes from other table changes. Keep that filtering tied to a clear outbox item shape or event marker.
Review the Lambda event-source mapping settings, including batch size, retry attempts, maximum record age, and a standard SQS destination for records discarded after processing failures. Consider partial batch responses: they let the function report failed records without unnecessarily retrying successful records from the same batch. DynamoDB Streams retains records for up to 24 hours, so retries and recovery need to fit within that window or use a separate reconciliation or replay source.
A stream record appears once in the stream, but Lambda may process it more than once when a batch is retried. Do not confuse stream storage behavior with end-to-end execution behavior.
Should the queue be Standard or FIFO?
| Queue type | Useful when | Design consequence |
|---|---|---|
| Standard | You want the Standard queue model and do not require FIFO ordering. | At-least-once delivery means consumers must tolerate duplicates. |
| FIFO | Ordering is a business requirement and queue-level deduplication is useful. | Queue-level deduplication does not remove the need to protect business effects against retries, crashes, or uncertain outcomes. |
Choose FIFO when ordering matters to the business, not as a substitute for consumer idempotency. AWS discusses FIFO queues for use cases where ordering matters in its SQS at-least-once delivery guidance.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How do you make your Lambda function idempotent?
Use the event ID as the idempotency key and make recognizing a duplicate consistent with performing the business effect. On a repeated delivery, the function should determine that the event has already been applied and avoid repeating that effect.
If the effect is a DynamoDB write, one robust pattern is to write an inbox or deduplication marker and apply the business update in the same DynamoDB transaction. A marker written separately from the effect has a crash window: the function could record the event as handled and fail before applying the change, or apply the change and fail before recording the marker.
For an effect outside DynamoDB, a local marker cannot by itself guarantee exactly-once behavior across systems. Prefer an idempotent downstream API when available, or maintain an inbox or processing ledger with a recovery plan for uncertain outcomes. If the consumer crashes after the external system accepts an operation but before the message is acknowledged, it may receive the event again; the downstream operation or reconciliation process must make that case safe.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallThe consumer should validate the event envelope and schema version, acknowledge or delete a message only after successful processing, and route poison messages through a deliberate dead-letter or redrive path. Make failures and recovery observable so an operator can distinguish a duplicate from an event that was never successfully applied.
How should you test retries and recovery?
Verify the failure paths before relying on the pipeline. AWS’s tutorial uses table writes, updates, and deletes and checks Lambda CloudWatch logs as a basic stream-trigger check; those checks do not by themselves prove that your end-to-end processing is duplicate-safe.
- Write a business item and its outbox event together, then confirm the relay publishes the expected event envelope to SQS.
- Cause the same event to be processed again and confirm its stable event ID prevents the business effect from being repeated.
- Submit a malformed or poison event and verify the intended failure, redrive, and monitoring behavior.
- Exercise an interruption around the relay’s SQS send attempt and inspect whether a retry can publish a duplicate; verify the consumer handles it safely.
- Trigger a discarded-record path and confirm the destination and recovery procedure work before the stream record ages out.
Transactions are regional: DynamoDB transaction atomicity applies within the Region of the write and does not extend across Regions through global-table replication. If the workflow spans Regions, design and test recovery for that boundary rather than assuming a cross-Region atomic transaction.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Recommended Free Tools
