What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Selective CI runs a subset of tests or workflows based on changed files and dependency information, aiming to reduce unnecessary CI work and shorten feedback time. But no verified results are available here for the experiments promised by “We Tested Selective CI on Real Open-Source Repos.” The available evidence explains how selection works and where it can fail; it does not establish a repository sample, savings figure, or miss rate for that purported test.
How does selective CI decide which tests to run?
A selector maps a proposed change to the tests or workflows it believes could be affected. The mapping may come from a build-system dependency graph, declared component ownership, or path rules. Selection is only as complete as that mapping: changed files must be connected to every relevant test, including tests affected through shared code and other inputs.
As an Amazon Associate I earn from qualifying purchases.
Test-target selection
Bazel provides mechanisms for identifying and running tests, including the tests() query operator for inspecting tests selected for a target. Its version 3.5.0 test encyclopedia describes test-suite expansion and other testing behavior, as well as parallel execution, sharding, and flaky-test reruns. These capabilities can help manage test execution, but they do not by themselves prove that a change-based selector will identify every test affected by a change. The documentation is version-specific; do not assume every implementation detail applies unchanged to current Bazel releases.
Tinder’s bazel-diff is an open-source approach for comparing Bazel targets across Git revisions to support test-target selection and selective building. Its existence is an implementation example, not evidence that it was used in the experiments implied by the title.
#1 Best Overall
- The FreeStyle log book includes sections for: Lunch, Dinner, Bedtime, Night
- Comments for each day of the week
- Log Book Dimensions L=4.25" x W=3.12" x H=0.12"
- Contains 5 book
Workflow selection
Selecting whole workflows is a coarser choice than selecting individual test targets. Composal’s vendor documentation says its workflow selection relies on declared component ownership and dependencies rather than inferring missing dependencies from imports. It also describes product-specific limits: the documented initial rollout is for supported native Composal-primary repositories and a top-only merge queue, with selection off by default and administrator enablement required. These are vendor-stated scope details, not independent validation of selection accuracy. See Composal’s Selective CI documentation.
Can selective testing miss a bug?
Yes. If the selector misses a dependency or relevant change, it can omit a test or workflow that would have caught a regression. A configuration can be syntactically valid and still describe the wrong coverage.
- Shared code: A change to a shared library may affect components that are not explicitly connected to it.
- Incomplete path rules: A file filter can omit relevant directories or configuration files.
- Generated code and lockfiles: Generated outputs, dependency manifests, and lockfiles may influence behavior without being represented in the selection model.
- Renames and deletions: A selector that considers only the new path, or only additions and modifications, may miss the impact of removed or renamed files.
- Uncertain diffs: If the system cannot establish what changed or how it maps to work, a narrow selection is not justified.
Composal recommends shadow-mode comparison with full workflow execution and says unknown or unproven diffs trigger all applicable workflows. Those are vendor-described safeguards; teams should verify how their own selector handles uncertainty and retain an independent safety net.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Does selective CI actually make CI faster?
It can reduce work when it safely skips irrelevant tests or workflows, but the size of any benefit depends on the project, selector, test suite, and CI setup. The evidence available for this topic does not establish an outcome from testing selective CI on open-source repositories.
A 2024 study by Shenyu Zheng, Bram Adams, and Ahmed E. Hassan examined Bazel build performance—not the accuracy or speed of change-based test selection. The authors gathered 383 GitHub Bazel projects, ran 3,500 experiments, and analyzed build logs for 70 buildable projects, including 102,232 incremental-build experiments over the last 100 commits of those projects. For long-build-duration projects, they reported median parallel-build speedups of 2.00x, 3.84x, 7.36x, and 12.80x at parallelism degrees 2, 4, 8, and 16, respectively. They also reported median incremental-build speedups of 4.22x with a build-system-independent CI cache and 4.71x with a build-system-specific cache, compared with clean builds for long-build-duration projects. These are results about build execution and caching, not proof that selective CI chooses tests correctly or produces those speedups. Read the study, “Does Using Bazel Help Speed Up Continuous Integration Builds?”, for its methods and qualifications.
To evaluate a selector in your own repository, measure both wall-clock feedback time and runner time. Include setup, retries, and fallback runs: a faster individual job does not necessarily mean lower total compute use, and a system that often falls back to full CI may deliver less time savings than its selection rate suggests.
How should a team validate selective CI?
Start by establishing what full CI costs and what the selector believes is affected. Keep full CI as the comparison baseline while validating the mapping, then broaden execution whenever the change or dependency plan is uncertain.
- Record a baseline: Measure full-CI wall-clock time and runner time, including setup, retries, and the workflows that normally run.
- Make dependencies explicit: Identify shared code, generated files, manifests, lockfiles, and other inputs that can affect tests or workflows.
- Run in shadow mode: Calculate the proposed selection while still running the full baseline. Compare selected work with the work full CI actually ran and investigate omissions.
- Handle uncertainty conservatively: If the diff, dependency mapping, or selection plan is unknown or unproven, run all applicable tests or workflows rather than silently narrowing coverage.
- Keep an independent safety net: Retain full-suite or equivalent coverage where needed, and monitor changes to dependency rules and repository structure.
- Reassess the trade-off: Compare saved feedback time and runner time against fallback frequency, retries, and the maintenance required to keep the dependency model accurate.
What results are established for the “we tested” claim?
No experiment results supporting that claim are established by the sources cited here. They do not identify the tested repositories, versions, selection configuration, baseline, measured savings, or missed-failure rate. The Bazel study supplies useful context about build performance, while Bazel and Composal documentation describe selection mechanisms and operational guidance; neither can stand in for selective-CI measurements on the claimed repository sample.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
