FREEDOM251

Freedom251 › Test Flakiness Management Software

Flaky Test Detective

Paid

Visit Flaky Test Detective Alternatives

Flaky Test Detective's own site

What Flaky Test Detective is

Flaky Test Detective is an open-source utility for analyzing repeated test runs and finding tests that fail inconsistently. It identifies timing variance, network errors and order-dependent failures, then can explain intermittent behavior and quarantine tests above a chosen threshold. The tool accepts Playwright JSON and JUnit XML, detecting the input format automatically. Reports can be printed in colorized console form, JSON, HTML with SVG charts or Markdown for GitHub pull-request comments. Trend history uses SQLite by default, with JSON available as a file-based fallback. The default configuration requires at least three runs before analysis. Quarantine mode tags tests over the threshold and outputs a pattern usable with Playwright’s --grep-invert option. Documentation covers an npm-installed command-line interface and programmatic use through an Analyzer import. A GitHub Actions example processes Playwright JSON results and uses the GitHub reporter. The repository lists an MIT license and identifies Quvantic as the builder. Its stated audience includes QA teams investigating flaky tests and their causes.

Who it suits

It suits QA teams using Playwright or JUnit results who need recurring flake detection, reports and quarantine workflows. Developers can use either the command line or Analyzer import.

What it does well

  • Supports Playwright JSON and JUnit XML
  • Detects timing, network and order failures
  • Provides console, JSON, HTML and PR reports
  • Tracks trends with SQLite or JSON
  • Offers threshold-based quarantine output

What to know first

  • Requires three runs by default
  • Focused on Playwright and JUnit formats
  • Quarantine depends on a configured threshold

Verdict

Flaky Test Detective provides open-source analysis, reporting and quarantine for recurring test failures. Its documented inputs and minimum-run requirement should fit teams with repeated Playwright or JUnit results.

Facts from Flaky Test Detective's own pages

CI integration
The README shows a GitHub Actions workflow that analyzes Playwright JSON results. · github.com, 11 Oct 2026
Detection
It detects timing variance, network errors, and order-dependent failures. · github.com, 4 Oct 2026
Flakiness patterns
It detects timing variance, network errors, and order-dependent failures. · github.com, 11 Oct 2026
History
Historical trend data can be stored in SQLite, with JSON available as a file-based fallback. · github.com, 11 Oct 2026
Intended users
The README describes QA teams as users who can use the tool to find flaky tests and investigate their causes. · github.com, 11 Oct 2026
License
The repository identifies its license as MIT. · github.com, 11 Oct 2026
Maker
The repository says it was built by Quvantic. · github.com, 4 Oct 2026
Minimum runs
The default configuration requires 3 runs before analysis. · github.com, 11 Oct 2026
Purpose
Flaky Test Detective analyzes test results across multiple runs to identify flaky tests and quarantine the worst offenders. · github.com, 11 Oct 2026
Quarantine
Quarantine mode tags tests above a threshold and outputs a pattern usable with Playwright's --grep-invert option. · github.com, 11 Oct 2026
Reports
Reporters include colorized console, JSON, HTML with SVG charts, and GitHub pull request comment markdown. · github.com, 11 Oct 2026
Supported formats
It supports Playwright JSON and JUnit XML parsers with automatic detection. · github.com, 11 Oct 2026
Test formats
It supports Playwright JSON and JUnit XML parsers, with automatic format detection. · github.com, 4 Oct 2026
Usage
The project documents installation through npm and provides a command-line interface and programmatic TypeScript usage. · github.com, 11 Oct 2026

Where Flaky Test Detective stands on Freedom251

Look at these too