Use pytest to organize readable tests, isolate setup, and check known examples; add Hypothesis when you can state a property that should hold across a defined range of inputs. Together, they can expose edge cases a happy-path review may miss—but they test only the requirements and input domains you define, so a passing suite is evidence, not a guarantee of correctness or security.
What pytest and Hypothesis each do
pytest is the test runner and organizing layer: it discovers test functions, runs assertions, provides fixtures, and supports finite sets of explicit cases. Its getting-started guide shows the basic install-and-test workflow, while the fixture guide explains reusable setup and teardown.
Hypothesis complements that structure by generating inputs from strategies and checking a property you specify. Its tests are ordinary Python tests that pytest can run. The Hypothesis quickstart demonstrates the approach and currently describes a default of 100 generated examples; confirm the defaults for the version installed in your project.
| Approach | Best suited to | Decision you must make |
|---|---|---|
| pytest assertions and parametrization | Known examples, regressions, and selected boundary cases | Which finite input/output pairs must be explicit? |
| Hypothesis property tests | Behavior expected to hold across a described input domain | What property should hold, and which inputs are valid? |
Install the packages and create a discoverable test
Install both packages in the project’s development environment and record them using the dependency-management tool the project already uses. The official pytest guide currently shows pip install -U pytest; Hypothesis’s quickstart shows pip install hypothesis. These are rolling documentation pages, not a promise that every version combination supports every Python release. Use versions compatible with the Python version your project and CI support.
#1 Best Overall
pytest automatically discovers conventional test modules and functions; its guide uses a file such as test_sample.py. Start with a test that names an observable behavior and asserts its expected result:
def test_adds_two_numbers():
assert add(2, 3) == 5Write the contract before adding tests for generated code. Identify expected outputs, valid inputs, error behavior, and relevant boundaries. Tests can check those decisions; they cannot make an unclear or incorrect requirement sound.
Use pytest for known examples and isolated setup
Parametrize the cases you already know
When several concrete inputs should produce specific outputs, @pytest.mark.parametrize keeps the examples together without obscuring them in a loop. pytest passes parameter values as-is, so avoid reusing mutable lists or dictionaries if a test might change them; one invocation could affect another. See pytest’s parametrization guide.
import pytest
@pytest.mark.parametrize(
"raw, expected",
[
("", None),
(" 42 ", 42),
],
)
def test_parse_known_cases(raw, expected):
assert parse_value(raw) == expectedKeep explicit rows for contractual examples, boundary values, and regressions for bugs already found. They make the intended behavior easy for another person to inspect.
Free tools Windows power users keep installed
One-click scans. No signup required.
Use fixtures to control resources
Fixtures make test dependencies explicit and let pytest manage setup and cleanup. Prefer the narrowest scope that fits the resource so unrelated tests do not share mutable state. For code that reads or writes files, request pytest’s tmp_path fixture to get a temporary directory associated with the test invocation. For environment variables, process state, or external services, use controlled fixtures or fakes rather than allowing a test to change a developer’s machine or a shared service.
def test_writes_output(tmp_path):
output = tmp_path / "result.txt"
write_result(output, "ready")
assert output.read_text() == "ready"Add Hypothesis when a property should hold across many inputs
Use @given with strategies that describe the valid input domain. A property test is not simply a large collection of random examples: the important work is defining a trustworthy rule and choosing inputs that satisfy the function’s preconditions. Hypothesis explores generated examples against that rule and can shrink a failure to a smaller counterexample.
from hypothesis import given, strategies as st
@given(st.integers())
def test_format_then_parse_round_trips(number):
assert parse_value(format_value(number)) == numberThis round-trip property is valid only if the functions’ contracts promise it for every integer. If formatting is intentionally lossy, or parsing supports only a narrower range, constrain the strategy to the supported domain or choose a different property. Do not narrow inputs so aggressively that meaningful boundary cases disappear.
Good property candidates
- Round trips: serializing then deserializing a supported value restores that value.
- Invariants: normalization preserves a required relationship, or a valid operation leaves an object in an allowed state.
- Reference comparisons: an optimized function agrees with a simpler, trusted implementation over inputs both support. Agreement is useful evidence, not proof that either implementation matches the requirement.
- Robustness within preconditions: valid inputs do not crash or violate a stated output contract.
- Stateful sequences: for code that changes state, test operation sequences only after specifying the allowed states and invariants.
If the requirement is one fixed output for one fixed input, a direct assertion may be clearer than a property test. If no trustworthy oracle exists, document that uncertainty rather than treating agreement between two implementations as correctness.
Combine explicit examples with generated cases
Known examples and generated cases answer different questions. Keep explicit pytest cases for recognizable requirements and regressions; use Hypothesis to search a defined domain for counterexamples to broader properties. Hypothesis also supports explicit examples alongside generated tests. A compact pattern might look like this:
import pytest
from hypothesis import given, strategies as st
@pytest.mark.parametrize(
"raw, expected",
[("", None), (" 42 ", 42)],
)
def test_parse_known_cases(raw, expected):
assert parse_value(raw) == expected
@given(st.integers())
def test_format_then_parse_round_trips(number):
assert parse_value(format_value(number)) == numberUse this only when both the concrete examples and the round-trip rule match the actual production contract. A generated test cannot validate a property that was guessed from how the code appears to work.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Make failures reproducible and tune test time
Hypothesis settings can control example counts, the example database, verbosity, and related test behavior. Its settings tutorial explains profiles and deterministic CI behavior. Keep the replay database available during normal development so a previously found failure can be reproduced; promote an important counterexample into a named regression example when that makes its significance clearer, while retaining the broader property.
Begin with a fast, repeatable required CI run. If broader exploration makes the suite too slow, a separate scheduled or opt-in job is a reasonable project choice—not a universal Hypothesis requirement. Choose settings for your runtime budget and risk, and check the installed version’s behavior rather than assuming documentation defaults never change.
Recommended Free Tools
What a passing suite can—and cannot—tell you
A failing test gives you a concrete mismatch between an asserted contract and observed behavior. A passing test says only that the tested cases and generated inputs did not reveal a mismatch under the properties and configuration you supplied. It cannot establish that requirements are complete, that an oracle is correct, or that generated code and its dependencies are secure.
Human review still needs to examine the requirements, test oracles, boundaries, error handling, dependency choices, and security-sensitive behavior. Neither pytest nor Hypothesis certifies AI-generated code as correct or safe, and no effectiveness percentage for this specific combination follows from the framework documentation.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
