Shifter

Free plan APILinuxSelf-hosted

What Shifter is

Shifter is an open-source container system for Linux high-performance computing. It lets users bring software environments to supercomputing facilities, with goals including portable deployments, repeatable science, and more efficient software delivery. Users commonly create environments with Docker; Shifter can retrieve images from Docker Hub or NERSC’s private registry. Its image gateway pulls registry images and repacks them, typically as squashfs, for compute nodes. A compute-node utility runs those environments, while example scripts and plugins connect Shifter with batch schedulers such as Slurm. Existing directories can be mounted into images for data or job output. NERSC documents an MPICH integration that inserts Cray libraries at runtime for multi-node communication when images meet compatibility requirements. Its loop-mount approach moves metadata work to compute nodes for workloads loading many shared libraries. Shifter uses privileged setup operations, then strips container privileges; documentation recommends avoiding privileged processes and using the newest version. The image gateway API requires Munge authentication. Shifter is free, with Linux, API, and self-hosted platforms. NERSC says it plans to phase it out eventually in favor of podman-hpc.

Who it suits

Shifter suits HPC users and administrators who need containerized environments on Linux supercomputing systems. It is aimed at teams using Docker images, batch schedulers, and multi-node workloads.

What it does well

Docker Hub and private registry image support
Scheduler integration examples, including Slurm
Mounts directories for data and job output
MPICH integration for compatible multi-node images
Free and open source

What to know first

Images above about 20 GB may time out
Requires Munge authentication for image gateway API
Uses privileged operations during container setup
Planned eventual phase-out for podman-hpc

Verdict

Shifter combines registry image conversion, compute-node execution, scheduler integration, and HPC-specific communication support. Its large-image limit, security requirements, and planned migration path matter when selecting it for new deployments.

Facts from Shifter's own pages

Architecture: The project includes a compute-node utility, an image gateway that pulls and repacks registry images, and example scripts or plugins for batch scheduler integration. · github.com, 11 Oct 2026
Authentication: Access to the image gateway API requires authentication using Munge. · github.com, 11 Oct 2026
Batch integration: The repository includes example integrations for workload managers, and NERSC's usage documentation shows Shifter jobs submitted with Slurm. · github.com, 11 Oct 2026
Container images: Users can create their software environments typically with Docker, and Shifter supports sharing HPC software through Docker and the Docker Hub community. · github.com, 11 Oct 2026
Contribution channel: The README directs contributors to a Slack channel and says code can be submitted through GitHub pull requests. · github.com, 11 Oct 2026
Current status: NERSC says it still supports Shifter but eventually plans to phase it out in favor of podman-hpc. · docs.nersc.gov, 11 Oct 2026
How it works: Shifter has a compute-node utility, an image gateway that pulls and repacks registry images, and example scripts or plugins for batch schedulers. · github.com, 11 Oct 2026
Image format: The image gateway repacks images into a format suitable for the HPC system, typically squashfs. · github.com, 11 Oct 2026
Image limits: NERSC says images larger than about 20 GB will likely time out when uploaded to Docker Hub and recommends contacting support for images over 20 GB. · docs.nersc.gov, 11 Oct 2026
Image registries: NERSC documentation says users can pull images from Docker Hub or NERSC's private registry, which the image manager converts to a flattened format for compute nodes. · docs.nersc.gov, 11 Oct 2026
Intended outcomes: The project aims to simplify software deployment, enable portable environments, support repeatable science, and improve system utilization by optimizing software delivery and I/O bottlenecks. · github.com, 11 Oct 2026
License: The repository license permits redistribution and use in source and binary forms subject to stated conditions and disclaimers. · github.com, 11 Oct 2026
MPI: NERSC documents an MPICH integration that swaps Cray libraries into images at runtime for multi-node communication, subject to image compatibility requirements. · docs.nersc.gov, 11 Oct 2026
Performance: NERSC says Shifter's loop-mount approach moves metadata operations to compute nodes and improves performance for workloads that load many shared libraries. · docs.nersc.gov, 11 Oct 2026
Purpose: Shifter adapts Linux container concepts for extreme-scale high-performance computing and lets users run their own software environments at supercomputing facilities. · github.com, 11 Oct 2026
Runtime privilege: The security documentation says container processes are stripped of privileges after launch and should not be able to regain them. · github.com, 11 Oct 2026
Scheduler integration: The project provides example scripts and plugins for integrating Shifter with batch scheduler systems. · github.com, 11 Oct 2026
Security controls: The project documentation describes nosuid mounts, restricting process capabilities to the null set, and attempting to apply Linux no_new_privs on newer systems. · github.com, 11 Oct 2026
Security guidance: The documentation recommends avoiding privileged processes in containers and using the most recent Shifter version. · github.com, 11 Oct 2026
Security model: Shifter uses privileged operations to set up containers; its security documentation says processes launched in a container have privileges stripped and should not be able to re-escalate. · github.com, 11 Oct 2026
Support: The project README directs users to its Google Groups email list for updates and community support. · github.com, 11 Oct 2026
Volume mounts: Users can mount existing directories into an image with a volume option, including paths for data and job output. · docs.nersc.gov, 11 Oct 2026

Where Shifter stands on Freedom251

Container Engines#24 of 31