Side by side
AgentDojo vs Augustus
Both stand on ai security testing tools. Which one you can start free with, which one lets you read the code, and what each costs once you pay.
Free plan APILinuxSelf-hosted
AgentDojo provides components for building and evaluating LLM agent security benchmarks, including custom attacks and defenses. Its changing API and incomplete result coverage are important constraints to account for.
Visit AgentDojo Free plan LinuxMacSelf-hosted+1
Augustus provides a broad set of model vulnerability probes, detectors, and report formats in a self-hosted package. It requires Go 1.27.0 or later, and should only be used on systems you own or are authorized to test.
Visit AugustusThe facts
| AgentDojo | Augustus | |
|---|---|---|
| Free plan | Yes | Yes |
| Open source | Not published | Not published |
| Runs on | API, Linux, Self-hosted | Linux, Mac, Self-hosted, Windows |
| Lowest paid plan | Not published | Free or on request |
| Plans published | None | 1 |
| Maker | Not published | Not published |
| Website | agentdojo.spylab.ai | github.com |
| On ai security testing tools | #10 of 29 | #6 of 29 |
AgentDojo: what it does well
- Free and self-hosted
- Supports custom attacks and benchmark tasks
- Pipelines can include defense components
- Run traces can be inspected in Invariant Explorer
What to know first
- Package API is still under development and may change
- Prompt injection detector requires the `transformers` extra
- Reported results do not cover every combination
Augustus: what it does well
- More than 210 probes across 47 categories
- Supports 28 provider categories and 43 variants
- More than 90 detectors
- Exports tables, JSON, JSONL, and HTML
- Free and Apache 2.0 licensed
What to know first
- Requires Go 1.27.0 or later
- API usage may require provider credentials
- Testing is limited to authorized systems
Alternatives to AgentDojo Alternatives to Augustus The whole list