HTCondor
HTCondor coordinates networked machines to run submitted computing jobs, including large batches such as thousands of runs across multiple data sets. It finds available machines, can use otherwise idle machines in a pool, and can restart a job elsewhere if its current machine becomes unavailable. ClassAds match machine resource offers with each job’s requirements and preferences. Users do not need login accounts on the machines running their jobs; file transfers and split execution support that access model. HTCondor can draw on clusters, cloud resources, and international grids, and flocking can connect multiple installations. It supports transfers to and from S3-compatible storage, including Google Cloud Storage through its S3 interoperability API, but not whole buckets or directories. The software is free under Apache License 2.0 and supports Linux, macOS, and Windows, as well as web, API, and self-hosted use. It includes Python bindings and user and administrator guides. Community support is available through a mailing list; fee-based contract support offers guaranteed resolution and faster turnaround.
Who it is for
HTCondor suits academic, government, and commercial organizations managing computational workloads across pools of networked machines. Its job scheduling and resource options may be useful for teams running repeated workloads over multiple data sets.
What is good
- Can restart jobs when a machine becomes unavailable
- ClassAds match machine offers to job needs
- Users need no login on execution machines
- Supports S3-compatible file transfers
- Includes Python bindings and administrator guides
What to know first
- Cannot transfer whole storage buckets or directories
- Freemium API keys do not access the API
- Contract support is fee-based
Verdict
HTCondor is suited to coordinating jobs across machines without requiring users to log in to each execution machine. Its storage-transfer limit and the need to purchase API access are important considerations.
Compared on GPU cluster management software
- Free plan
- Yes
- Deployment model
- self_hosted
- Workload scheduling
- both
- Kubernetes support
- Yes
- Quota controls
- Yes
- GPU utilization metrics
- Yes
- Cloud GPU support
- Yes
