Machinery

What Machinery is

Machinery is a free, self-hosted Go library for asynchronous task and job queues using distributed message passing. Worker processes consume queued tasks, with each task handled in a goroutine. Tasks can be delayed with ETA timestamps, scheduled periodically, or arranged into groups, chains, and chords with success and error callbacks. Configurable retries space requests using a Fibonacci sequence. Message brokers include AMQP, Redis, AWS SQS, and Google Cloud Pub/Sub, while result backends include Redis, Memcache, AMQP, and MongoDB. Results expire after 3600 seconds by default. Pending-task inspection is currently available only with Redis as the broker, and the README cautions against AMQP as a result backend for large numbers of parallel tasks. Configuration comes from environment variables or YAML, with optional live reloading every 10 seconds. Task messages and results are JSON encoded, limiting task data to types with native JSON representations. Machinery is licensed under MPL-2.0.

Who it suits

Machinery suits Go developers who need self-hosted asynchronous queues, scheduled jobs, retries, or workflows. It is relevant when you can operate worker processes and select supported brokers and result stores.

Limits

Task results expire after 3600 seconds by default. Pending tasks can currently be inspected only when Redis is the broker, and task messages and results must use native JSON representations.

Getting started

1. Install the library with go get
2. Configure a supported message broker
3. Choose a result backend
4. Load settings from environment variables or YAML
5. Run worker processes
6. Define queued, scheduled or workflow tasks

Verdict

Machinery provides a broad queue and workflow feature set in a self-hosted Go library with no listed cost. Broker, result-backend, inspection, and JSON constraints should shape deployment choices.