FREEDOM251

Freedom251 › AI Security Testing Tools

RAMPART

Free plan APILinuxMacWindows

Visit RAMPART Alternatives

RAMPART's own site

What RAMPART is

RAMPART is a free, open-source framework for writing and running safety and security tests for agentic AI applications. It works within pytest: users connect an agent with an adapter, while RAMPART handles execution strategies, evaluation, and reporting. Its attack tests assess whether an attack is detected, with detection marked UNSAFE; probes check for expected behavior, with a detected expected outcome marked SAFE. The documented attack test checks cross-prompt injection (XPIA), including whether malicious material placed in data sources can influence an agent when retrieved. Its built-in probe checks expected responses, tool use, or side effects, making probes useful for regression testing. Outcomes can be SAFE, UNSAFE, UNDETERMINED, or ERROR, with trace and evaluation details. RAMPART registers as a pytest plugin automatically and includes harm and trial markers. It creates structured JSON reports, supports custom report sinks, and can combine output from parallel pytest-xdist workers. Installation requires Python 3.11 or later and uses uv or pip on Linux, macOS, or Windows. PyRIT is a dependency for LLM interaction, prompt normalization, and conversation memory; an optional OneDrive extra enables its built-in OneDriveSurface for placing XPIA payloads in OneDrive. The project is MIT-licensed and identifies Microsoft Corporation as copyright holder.

Who it suits

RAMPART suits developers building agentic AI applications who want to write safety and security tests within pytest. It is also relevant to teams using attack tests, behavior probes, and JSON reporting in their test workflows.

What it does well

  • Free and MIT-licensed.
  • Runs within pytest and registers automatically as a plugin.
  • Includes a cross-prompt injection attack test and behavior probe.
  • Reports safety status, traces, and evaluation details.
  • Supports parallel pytest-xdist runs with a unified report.
  • Offers JSON reports and hooks for custom report sinks.

What to know first

  • Requires Python 3.11 or later.
  • Users must provide an adapter connecting their agent.
  • Teardown-only results are excluded from xdist report streaming.
  • Mixed RAMPART versions across xdist controller and workers are unsupported.

Getting started

  1. Open the RAMPART website for installation instructions.
  2. Install Python 3.11 or later on Linux, macOS, or Windows.
  3. Use uv or pip to install RAMPART and its PyRIT dependency.
  4. Provide an adapter connecting your agent to RAMPART.
  5. Add attack tests or behavior probes to your pytest workflow.
  6. Install the optional onedrive extra to use the built-in OneDriveSurface.

Verdict

Choose RAMPART if you want a pytest-native way to test agent safety, including cross-prompt injection and expected behavior. It is a fit for teams comfortable supplying an agent adapter and managing a Python installation; account for its documented xdist restrictions when planning parallel reporting.

Questions

Does RAMPART cost anything?

RAMPART is free and open source under the MIT license.

Which platforms does it support?

Its installation instructions cover Linux, macOS, and Windows. Its listed platforms also include API.

What tests are included?

The documented built-in attack test checks cross-prompt injection. A built-in probe verifies expected agent behavior, such as responses, tool use, or side effects.

Does it need an integration with my agent?

Yes. Users provide an adapter connecting their agent to RAMPART.

What does RAMPART use PyRIT for?

PyRIT supports LLM interaction, prompt normalization, and conversation memory, and is installed as a dependency.

Can I use OneDrive for XPIA payloads?

An optional onedrive extra installs dependencies for using the built-in OneDriveSurface to place XPIA payloads in OneDrive.

Facts from RAMPART's own pages

Attack and probe tests
Its two test categories are attacks, where a detected attack means UNSAFE, and probes, where detected expected behavior means SAFE. · microsoft.github.io, 8 Oct 2026
Attack testing
Its XPIA tests check whether malicious content planted in data sources can manipulate an agent when retrieved. · microsoft.github.io, 9 Oct 2026
Behavior probes
Behavioral probes check for expected responses, tool use, or side effects and are described as useful for regression testing. · microsoft.github.io, 9 Oct 2026
Evaluation output
Results include a safety status of SAFE, UNSAFE, UNDETERMINED, or ERROR, along with trace and evaluation details. · microsoft.github.io, 9 Oct 2026
Included tests
The documented built-in attack is cross-prompt injection (XPIA), and the documented probe verifies expected agent behavior. · microsoft.github.io, 8 Oct 2026
Install requirements
The installation guide requires Python 3.11 or later and supports installation with uv or pip. · microsoft.github.io, 9 Oct 2026
Integration model
Users provide an adapter connecting their agent to RAMPART, which supplies execution strategies, evaluators, pytest integration, and reporting. · microsoft.github.io, 8 Oct 2026
Intended users
The project describes its framework as a developer-friendly way to write and run safety and security tests for AI agents. · github.com, 9 Oct 2026
License
The GitHub repository identifies RAMPART as MIT-licensed. · github.com, 9 Oct 2026
Limitations
The xdist documentation says results recorded only during fixture teardown are excluded from report streaming and mixed RAMPART versions across controller and workers are unsupported. · microsoft.github.io, 8 Oct 2026
Maker
The documentation identifies Microsoft Corporation as its copyright holder. · microsoft.github.io, 8 Oct 2026
OneDrive
An optional onedrive extra installs dependencies for using the built-in OneDriveSurface to place XPIA payloads in OneDrive. · microsoft.github.io, 8 Oct 2026
Optional integration
The optional onedrive extra adds dependencies for using the built-in OneDriveSurface to place XPIA payloads in OneDrive. · microsoft.github.io, 9 Oct 2026
Parallel execution
RAMPART supports pytest-xdist parallel runs and produces a unified report across worker processes. · microsoft.github.io, 8 Oct 2026
Purpose
RAMPART is a pytest-native safety testing framework for agentic AI applications. · microsoft.github.io, 9 Oct 2026
PyRIT
RAMPART uses PyRIT for LLM interaction, prompt normalization, and conversation memory, and installs PyRIT as a dependency. · microsoft.github.io, 8 Oct 2026
PyRIT integration
RAMPART uses PyRIT for LLM interaction, prompt normalization, and conversation memory, and installs PyRIT as a dependency. · microsoft.github.io, 9 Oct 2026
pytest integration
RAMPART registers as a pytest plugin automatically and provides harm and trial markers. · microsoft.github.io, 9 Oct 2026
pytest plugin
RAMPART registers automatically as a pytest plugin through the pytest11 entry point, without requiring conftest.py activation configuration. · microsoft.github.io, 8 Oct 2026
Python requirement
Installation requires Python 3.11 or later and uses uv or pip for package management. · microsoft.github.io, 8 Oct 2026
Reporting
RAMPART provides structured JSON reporting, including a built-in JSON file report sink and a hook for custom sinks. · microsoft.github.io, 8 Oct 2026
Reporting and CI
Built-in JSON report sinks and pytest-xdist support let teams collect reports for CI dashboards across parallel workers. · microsoft.github.io, 9 Oct 2026
Security boundary
For xdist worker serialization, RAMPART documents JSON-safe payloads, rejection of unknown schema versions, a default 16 MiB per-result cap, and ANSI escape stripping. · microsoft.github.io, 8 Oct 2026
Supported installation systems
The installation instructions provide commands for Linux, macOS, and Windows. · microsoft.github.io, 8 Oct 2026

Where RAMPART stands on Freedom251

Look at these too