DriversRecommendedOutdated drivers can make a good PC feel brokenScan driver issues before chasing fixes manually.Scan NowOctober DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PC×
Skip to content
World desk4 min

Decoupled Distributed Queues and Pub/Sub in Python with Redis

Redis Pub/Sub broadcasts transient messages to current listeners; a Redis-backed queue stores job state so workers can claim, retry, and recover tasks. Learn which pattern fits and how redis-py subscriptions work.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Use Redis Pub/Sub when a message should reach the subscribers listening now; use a Redis-backed queue when workers must claim, retry, and recover jobs. They both decouple producers from consumers, but they do not provide the same delivery guarantees. The official material relevant to this topic documents Redis and redis-py, not a distinct Python package called WRedis, so this guide uses the documented Redis APIs and patterns.

How Redis Pub/Sub and a work queue differ

Redis Pub/Sub lets publishers send messages to channels without addressing particular receivers. Subscribers receive messages for channels they follow, in publish order. Redis describes this separation as enabling “greater scalability and a more dynamic network topology.” Redis Pub/sub documentation

That decoupling does not make Pub/Sub a job queue. Redis Pub/Sub is at-most-once: if a subscriber is offline or cannot process a message, Redis does not retain it for that subscriber to retrieve later. Redis Streams, by contrast, persist messages and support at-least-once delivery. A work queue can also maintain job state and recovery behavior for workers.

Need Redis Pub/Sub Redis-backed queue or Streams
Work shape Broadcast an event to subscribers currently listening. Hand work to workers for processing.
Consumer offline Missed messages are not replayed. A queue can retain job state and reclaim timed-out work; Streams persist messages and support at-least-once delivery.
Typical use Live notifications, cache invalidation, or UI updates. Background work that needs retries, status, or recovery.
Trade-off Simple, low-latency fan-out, but transient delivery. More state and recovery logic in exchange for stronger job-handling behavior.

How to use Redis Pub/Sub in Python

With redis-py, publishing uses the Redis client, while subscriptions use a separate PubSub object. Redis’ Python example lists Redis 6.2 or later, Python 3.9 or later, and redis-py 5.0 or later as requirements for that example; these are not universal minimum requirements for every Redis or Pub/Sub implementation. Redis pub/sub with redis-py

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Subscribe to a channel

Create a PubSub object, subscribe to a channel, and read messages from that subscription. The consuming process must remain connected and listening to receive live events. Publishing can happen through a Redis client independently of which subscribers are connected.

Use pattern subscriptions when appropriate

The documented example also demonstrates glob-style pattern subscriptions, which let a subscriber listen for channel names matching a pattern rather than subscribing to each exact channel individually. Choose exact channels when recipients need a specific topic; use patterns only when matching a family of channel names is part of the design.

The example’s in-process buffer of recent messages is only for inspecting messages during the demo. It is not Redis persistence and does not change Pub/Sub’s at-most-once behavior.

How to build a Redis-backed job queue in Python

A queue needs to represent more than a message being broadcast. Redis’ Python job-queue guide describes storing job metadata and state in Redis data structures, allowing worker processes to claim work, record outcomes, retry failures, and reclaim work that remains processing beyond a visibility timeout. Redis job queue with redis-py

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

What the queue design tracks

  • Job metadata and state: stored as Redis hashes, with pending and processing lists used to organize work.
  • Claims: workers move jobs into processing through atomic claims so multiple workers do not simply take the same pending item.
  • Outcomes: completion and failure history provides a record of how jobs ended.
  • Retries and recovery: failed jobs can be retried, while a visibility-timeout sweeper can reclaim work that appears stuck.

These are design details of Redis’ documented example, not guarantees supplied automatically by every queue built on Redis. The queue implementation must define its state transitions, retry policy, and timeout handling. The same guide lists Redis 6.2 or later, Python 3.9 or later, and redis-py 5.0 or later for its example.

Where Pub/Sub fits in a queue

The documented queue example uses Pub/Sub for completion notification, while Redis data structures hold job metadata and state. That split is useful: a notification can tell an interested client that something happened, but the job’s stored state—not the notification—is what supports inspection and recovery.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Using redis-py with asynchronous consumers

For async code, redis-py’s documented pattern is to await subscription setup and then consume the listener as an async iterator. Give each consuming task its own PubSub object rather than sharing one subscription object among concurrent tasks. Asynchronous operations with redis-py

The async interface changes how the application waits for messages; it does not turn Pub/Sub into durable delivery. Consumers that need messages retained while disconnected should choose a persistent mechanism such as Redis Streams or a queue design with stored state and recovery.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Choose based on what happens when a consumer is away

  • Choose Pub/Sub when the event is useful only to currently connected listeners and missing it is acceptable—for example, a live UI update or a cache-invalidation signal backed by another way to refresh state.
  • Choose a queue when a worker must eventually process a job, and the application needs explicit job status, retries, or reclamation after a worker stops responding.
  • Consider Redis Streams when persisted messages and at-least-once delivery are the relevant requirements, rather than transient broadcast.

Plan for duplicate handling wherever a design retries or provides at-least-once delivery: a worker may encounter work again, so the application’s processing behavior should tolerate that possibility. Keep the guarantees of the chosen implementation distinct from the guarantees of the underlying Redis feature.

What “WRedis” means here

The title’s WRedis wording does not identify a documented package or API in the official sources covered here. Redis’ Python client is redis-py; its repository and documentation describe the client and the examples above. redis-py: Redis Python client If WRedis refers to a different project, its package identity and API are not established by these sources, so do not assume that its interfaces match redis-py.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Windows Errors? Fix Them Before They SpreadFree repair scan
Outdated Drivers Are Slowing You DownFree scan - exact matches

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.