Freedom251 › Mutation Testing Tools
pytest-gremlins
What pytest-gremlins is
pytest-gremlins is a pytest plugin for checking whether a Python project's tests detect small code changes. It introduces mutations and runs tests against them, helping teams spot mutations their tests leave undetected. The getting started guide requires Python 3.11 or later, pytest 7.0 or later, and an existing test suite. Installation is documented with pip, uv, and Poetry; a basic run uses `pytest --gremlins`. To manage execution, it offers mutation switching, coverage-guided test selection, incremental analysis, and parallel execution, and works with pytest-xdist. Results can be shown in the console or saved as HTML or JSON, with multiple report formats in one run. Configuration is available through `pyproject.toml` or command-line flags, and inline pragmas can pardon mutations. A GitHub Action can run checks and enforce a score threshold; CI examples also cover GitLab CI, CircleCI, and Dagger. The guide warns that a first run may take 10–100 times as long as a normal test run, though caching may shorten later runs. The package is MIT-licensed and classified as Alpha.
Who it suits
It suits Python teams using pytest that want to assess tests around business logic, security-critical code, or financial calculations. It requires Python 3.11 or later, pytest 7.0 or later, and an existing test suite.
What it does well
- Free, MIT-licensed package.
- Supports incremental analysis and parallel execution.
- Offers console, HTML, and JSON reports.
- GitHub Action can enforce a mutation-score threshold.
What to know first
- Requires Python 3.11 or later and pytest 7.0 or later.
- A first run may take 10–100 times as long as normal tests.
- Development status is classified as Alpha.
Limits
A first mutation-testing run may take 10–100 times as long as a normal test run, although caching can make later runs faster. Equivalent mutations can make a 100% mutation score impossible and not worth pursuing.
Plans
| Plan | Billed and limits | Price |
|---|---|---|
| pytest-gremlins | MIT-licensed Python package · requires Python 3.11+ | Free |
As github.com lists them, read 7 Oct 2026. The maker's own page has the final word.
Getting started
- Install Python 3.11 or later and pytest 7.0 or later in a project with existing tests.
- Install pytest-gremlins with pip, uv, or Poetry.
- Run `pytest --gremlins` to start mutation testing.
- Configure options in `pyproject.toml` or with command-line flags.
- Optionally use pytest-xdist or the GitHub Action to include runs in CI.
Verdict
pytest-gremlins adds mutation testing to existing pytest suites, with reporting and CI options. Allow for a potentially much longer first run, and do not treat a 100% mutation score as a necessary target: equivalent mutations can make it unattainable.
Questions
Is pytest-gremlins free?
Yes. It is a free, MIT-licensed Python package.
What does it require?
It requires Python 3.11 or later, pytest 7.0 or later, and a project with existing tests.
Which platforms does it support?
Linux is listed as a supported platform.
What report formats are available?
It supports console, HTML, and JSON reports, including multiple formats in one run.
Can it run in CI?
The project provides a GitHub Action that can enforce a mutation-score threshold, plus manual CI examples for GitHub Actions, GitLab CI, CircleCI, and Dagger.
What license does it use?
The project is MIT licensed; the license provides the software “AS IS” without warranty.
Facts from pytest-gremlins's own pages
- Audience
- The guide recommends focusing mutation testing on business logic, security-critical code, and financial calculations. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- CI integrations
- The GitHub Action documentation points to manual CI workflow examples for GitHub Actions, GitLab CI, CircleCI, and Dagger. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- CI options
- The CI integration guide covers GitHub Actions, GitLab CI, CircleCI, and Dagger. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- Configuration
- Options can be configured through `pyproject.toml` or command-line flags, and mutations can be pardoned with an inline pragma. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- Development status
- The package metadata classifies its development status as Alpha. · pypi.org, 7 Oct 2026
- GitHub integration
- The composite GitHub Action installs a project, runs mutation testing, and can fail a workflow when the mutation score is below a threshold. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- Installation
- The project documents installation with pip, uv, and Poetry. · github.com, 7 Oct 2026
- Installation requirements
- The getting started guide requires Python 3.11 or later, pytest 7.0 or later, and a project with existing tests. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- Integrations
- The documentation describes compatibility with pytest-xdist and a GitHub Action for running mutation tests and enforcing a score threshold. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- License
- The project is MIT licensed. · github.com, 7 Oct 2026
- Limits
- The guide says equivalent mutations can make a 100% mutation score impossible and not worth pursuing. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- Mutation control
- Users can suppress equivalent or untestable mutations inline with `# gremlin: pardon[reason]`. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- Publishing provenance
- PyPI reports that the October 6, 2026 release was uploaded using Trusted Publishing and has a verified attestation signed by GitHub Actions. · pypi.org, 7 Oct 2026
- Purpose
- pytest-gremlins is a pytest plugin that injects small code mutations and checks whether tests catch them. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- pytest integration
- The plugin works with existing pytest test suites and pytest-xdist, including `-n auto`. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- Python requirement
- The package requires Python 3.11 or later. · pypi.org, 7 Oct 2026
- Reports
- It supports console, HTML, and JSON reports, including multiple report formats in one run. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- Runtime
- The guide says a first mutation-testing run may take 10–100 times as long as a normal test run; caching can make later runs faster. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- Security and license
- The repository identifies the project as MIT-licensed; the license text provides the software “AS IS” without warranty. · github.com, 7 Oct 2026
- Setup
- The project says mutation testing can be started with `pip install pytest-gremlins` and `pytest --gremlins`. · github.com, 7 Oct 2026
- Speed
- Its speed features include mutation switching, coverage-guided test selection, incremental analysis, and parallel execution. · pytest-gremlins.readthedocs.io, 7 Oct 2026
- Speed features
- It uses mutation switching, coverage-guided test selection, incremental analysis, and parallel execution. · pytest-gremlins.readthedocs.io, 7 Oct 2026
Where pytest-gremlins stands on Freedom251
- Mutation Testing Tools#21 of 25