FREEDOM251

Freedom251 › Compare

Side by side

AgentDojo vs Augustus

Both stand on ai security testing tools. Which one you can start free with, which one lets you read the code, and what each costs once you pay.

AgentDojo

Free plan APILinuxSelf-hosted

AgentDojo provides components for building and evaluating LLM agent security benchmarks, including custom attacks and defenses. Its changing API and incomplete result coverage are important constraints to account for.

Visit AgentDojo

Augustus

Free plan LinuxMacSelf-hosted+1

Augustus provides a broad set of model vulnerability probes, detectors, and report formats in a self-hosted package. It requires Go 1.27.0 or later, and should only be used on systems you own or are authorized to test.

Visit Augustus

The facts

AgentDojoAugustus
Free planYesYes
Open sourceNot publishedNot published
Runs onAPI, Linux, Self-hostedLinux, Mac, Self-hosted, Windows
Lowest paid planNot publishedFree or on request
Plans publishedNone1
MakerNot publishedNot published
Websiteagentdojo.spylab.aigithub.com
On ai security testing tools#10 of 29#6 of 29

AgentDojo: what it does well

  • Free and self-hosted
  • Supports custom attacks and benchmark tasks
  • Pipelines can include defense components
  • Run traces can be inspected in Invariant Explorer

What to know first

  • Package API is still under development and may change
  • Prompt injection detector requires the `transformers` extra
  • Reported results do not cover every combination

Augustus: what it does well

  • More than 210 probes across 47 categories
  • Supports 28 provider categories and 43 variants
  • More than 90 detectors
  • Exports tables, JSON, JSONL, and HTML
  • Free and Apache 2.0 licensed

What to know first

  • Requires Go 1.27.0 or later
  • API usage may require provider credentials
  • Testing is limited to authorized systems

Alternatives to AgentDojo Alternatives to Augustus The whole list