Side by side
Csmith vs EvoMaster
Both stand on fuzz testing software. Which one you can start free with, which one lets you read the code, and what each costs once you pay.
Free plan LinuxWindows
Choose Csmith when you need locally generated C programs for compiler comparisons or test-suite creation. Timely maintainer responses are not assured.
Visit Csmith Free plan APILinuxMac+2
EvoMaster provides free fuzz-based test generation across several API types and programming languages. Account for the white-box driver requirement and the lack of automatic external-service mocking when planning its use.
Visit EvoMasterThe facts
| Csmith | EvoMaster | |
|---|---|---|
| Free plan | Yes | Yes |
| Open source | Not published | Not published |
| Runs on | Linux, Windows | API, Linux, Mac, Self-hosted, Windows |
| Lowest paid plan | Free or on request | Free or on request |
| Plans published | 1 | 1 |
| Maker | Not published | Not published |
| Website | github.com | github.com |
| On fuzz testing software | #9 of 26 | #5 of 26 |
Csmith: what it does well
- Generates C programs without undefined behavior.
- Includes statistics with each generated program.
- Supports comparing GCC and Clang output.
- Command-line options customize program generation.
- Runs locally on Linux and Windows.
What to know first
- Generated programs may contain infinite loops and need timeouts.
- Building from source on Ubuntu requires g++, CMake, and m4.
- Maintainer responses may be delayed.
EvoMaster: what it does well
- Targets REST, GraphQL, and RPC APIs.
- Generates tests in Java, Kotlin, Python, and JavaScript.
- Uses more than 30 oracles to check for faults.
- Can run in GitHub Actions.
- Project says it has no telemetry.
What to know first
- White-box testing requires a manually written driver.
- Does not yet automatically mock external services.
- REST APIs need an OpenAPI schema.
- OpenAPI 3.2 support is on hold.
Alternatives to Csmith Alternatives to EvoMaster The whole list