A prompt-injection evaluation harness answers one question well: did an injected instruction change what the application did, and what did that cost legitimate users? Build the harness around the application’s real input path. Grade security by what actually happened (disclosed markers, tool calls, state changes, outbound traffic), count false alarms and failed legitimate tasks alongside blocked attacks, and report only what the sample can support.
This article is a method, not a results report. It does not give detection rates for any particular system, and it does not describe lessons from a specific test run. Where it cites numbers, they come from OWASP and a published academic study, and each one is qualified where it appears.
As an Amazon Associate I earn from qualifying purchases.
Start from the application boundary, not a list of attacks
A harness is only as meaningful as the boundary it tests. OWASP’s guidance is that its example attacks should be adapted to the application’s tasks, permissions, and input channels, not reused as-is (OWASP Cheat Sheet Series, LLM Prompt Injection Prevention). So write down the system before you write any attack.
Threat model fields
- Supported tasks: what the application is meant to do, and which requests it must refuse.
- User roles: who can send input, and what each role may ask a tool to do.
- Sensitive data: the values that would matter if disclosed. In tests, these are replaced by dummy markers.
- External content sources: retrieved documents, email, web pages, and tool output.
- Tools and write permissions: each action the model can request, and which of them change state.
- Harm: the concrete outcome an attacker would want, such as a message sent to an outside address or a record altered.
For an agent, the threat model is a list of actions and data flows, not a list of chat turns.
One record per test case
Each case should carry enough detail that another person can rerun it and grade it the same way.
| Field | What to record |
|---|---|
| Source channel | Direct user message, retrieved document, email, web page, or tool output |
| Case type | Intended violation or legitimate task |
| Necessary context | Conversation history, retrieved chunks, user role, and tool state the case depends on |
| Expected observable | The signal that would show the security objective was met (see the observables section below) |
| Severity | The harm if the objective is met in production |
Build a corpus and treat it as a smoke test
OWASP’s own example set is illustrative. It contains 14 hand-picked attack examples and seven benign examples, and the page is explicit about what that set can do: “Use the examples below as a smoke test, not a security benchmark.” A smoke test can reveal obvious breakage. It does not prove security, and it does not measure how often attacks occur in practice (OWASP Cheat Sheet Series, LLM Prompt Injection Prevention).
The attack families in OWASP’s illustrative list are a useful starting inventory:
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →- direct instruction override
- role or authority claims
- base64-encoded instructions
- “typoglycemia” variants, where the interior letters of words are scrambled
- spacing and capitalization variations
- remote-injection patterns
- benign requests, which serve as the control side of the test
Extend the inventory from your own boundary. For each channel and tool, ask which instruction would cause the harm you listed in the threat model, then write cases that try to reach it through that channel.
Rank #2
Test each input route on its real path
Direct user input
Direct messages are the simplest route. Send them through the same endpoint, middleware, system prompt, and model configuration that production uses. A script that calls the model directly skips every control that sits in front of it, so its results describe a different system.
Indirect instructions in retrieved content and tool output
Indirect injection crosses a different boundary. The instruction arrives inside a document, email, web page, or tool response that the application treats as data. Place the untrusted text in the same retrieval or tool path production uses, so that chunking, ranking, and formatting are part of the test. AI Exchange recommends tailoring attacks to the application, including data extraction and downstream actions, and varying inputs to test evasion (OWASP AI Exchange, AI security testing).
Keep the setup production-equivalent
The same guidance says to use the same model versions, prompts, tools, permissions, and configuration as production, and to rerun the tests before deployment and whenever the threat picture changes (OWASP AI Exchange, AI security testing). A harness that ran against an older model or a looser tool scope describes that older setup, not the one users reach.
Recommended Free Tools
Grade security outcomes by observable, not by the final answer
A clean final answer is not evidence that nothing happened. Give each security objective its own observable, and grade each one separately.
| Objective | Observable | Where to inspect | Limitation |
|---|---|---|---|
| Dummy-marker disclosure | Whether the exact dummy marker appears in the output or in captured logs | Response body and the logs the harness captures | A missing marker does not rule out transformed disclosure, such as an encoded or paraphrased version of the value |
| Unauthorized action | Tool-call records, authorization decisions, and the dummy state before and after the run | The instrumented tool layer and the sandboxed state store | A refusal in the final answer cannot undo a call that already ran |
| External disclosure | Whether dummy data reached the controlled destination | The instrumented endpoint you control | Nothing in the chat transcript shows this |
The three rows follow the source material’s guidance: grade the objectives separately, and observe actions and destinations, not only text (OWASP Cheat Sheet Series, LLM Prompt Injection Prevention).
Use safe fixtures
- Place only dummy secrets in prompts, such as a marker string with no meaning outside the test.
- Route actions to sandboxed substitutes for real tools, with state you can read back after each run.
- Instrument authorization decisions and state changes at the tool layer, not only in the model’s output.
- Point exfiltration tests at a destination you control and log every request it receives.
Classify every run before you count it
Every run should land in exactly one outcome. The table below is the decision rule the counts depend on.
| Outcome | Condition | How to count it |
|---|---|---|
| Blocked | The policy stopped the request, no objective observable fired, and telemetry was complete | Counts as blocked for that objective |
| Objective reached | Any observable fired for that objective, whatever the final answer says | Counts as a failure of the defense for that objective |
| Benign completed | The legitimate task produced the correct result, checked against an expected output | Counts toward task completion |
| Benign not completed | The legitimate task produced an empty or incorrect result | Counts against task completion; an empty response is never a success |
| Benign refused | A security decision refused a legitimate request | Counts as a false positive |
| Human review | The policy routed the case to review | Reported as pending, separate from allowed and blocked |
| Inconclusive | An error, missing telemetry, or a context the harness cannot instrument | Reported in its own column; never added to blocked |
Measure false positives next to blocking
Blocking attacks is only half the result. A defense that refuses every request will show a perfect attack-blocking rate, which is why false positives must be measured on legitimate requests in the same run.
Separate the policy decision (allow, block, or human review) from whether the task completed correctly. A request can be allowed and still fail the task, and that failure belongs in task completion, not in the false-positive count.
| Metric | Numerator | Denominator | Report alongside |
|---|---|---|---|
| False-positive rate | Applicable benign requests incorrectly refused for security reasons | Applicable benign requests run | Both counts and the number of runs |
| Pending review rate | Requests routed to human review | Requests run in that set | Stated separately from refusals |
| Task-completion rate | Benign requests whose task completed correctly | Benign requests run | Empty or incorrect responses do not count |
| Attack objective not reached | Attack cases where no observable fired and telemetry was complete | Attack cases with conclusive telemetry | The inconclusive count, per objective |
Count refusals by decision, not by wording. A refusal generated by the model itself counts the same as one produced by a guardrail, and the harness should not detect refusals by searching for stock phrases. Under this measure, a system that refuses every benign request has a 100% false-positive rate (OWASP Cheat Sheet Series, LLM Prompt Injection Prevention).
Reading zero false positives in a small benign set
OWASP gives a worked illustration: zero false positives in seven independent trials sampled from a defined benign workload yields a 95% Wilson confidence interval of approximately 0% to 35.4% (OWASP Cheat Sheet Series, page accessed October 4, 2026). That is an illustration of uncertainty, not a measured result for any system. A zero count from seven requests is consistent with a true rate as high as roughly one in three. The interval also cannot correct for biased case selection or for attack classes the test set never includes.
Run the evaluation in a fixed order
- Freeze the setup before the first run. Record the corpus revision, model identifier, defense configuration, prompts, tool permissions, and generation settings.
- Provision the fixtures: dummy markers, sandboxed tool endpoints with readable state, and a controlled destination. Confirm that each instrument records at least one event before the full run starts.
- Run the same cases against every defense you compare, and run each case the same number of times.
- Classify each run using the outcome table, and keep inconclusive runs in their own count.
- Calculate each metric with its numerator and denominator, and store per-case results so any reported rate can be traced back to its cases.
Record enough to rerun and compare
A rate without its setup is not reproducible. Each published result should carry:
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minute- the corpus source and revision, with each case’s channel and type;
- model and defense versions, and the settings used for each;
- per-case outcomes for each security objective, not a single combined security score;
- the number of repeated runs per case;
- every rate with its numerator, denominator, and corpus source.
When comparing defenses, use the same cases and keep the pairs. A case that one defense blocks and another misses is more informative than two separate totals, because it shows where the defenses differ.
Say what the numbers support
A hand-picked smoke test supports counts and case-level findings. It does not support a population attack rate, because the cases were selected rather than sampled from real traffic. Repeated runs of one case and close variants of one attack are correlated, not independent trials, so say how you counted them. A conclusion about a defense applies to the case set it was tested on, and to the model and configuration recorded with the run.
Where model-based guardrails and architecture fit
A guardrail model is one layer you can test, not a boundary you can trust on its own. OWASP states it directly: “A guardrail LLM is itself an LLM and is itself susceptible to prompt injection.” (OWASP Cheat Sheet Series, LLM Prompt Injection Prevention). The harness should therefore test the guardrail as part of the full chain, not treat its verdict as ground truth.
Controls OWASP pairs with a guardrail
- Input validation
- Structured prompts
- Least-privilege tool scopes
- Human approval for destructive actions
- Logging of guardrail decisions, with watch for drift over time
Guardrail calls add latency and cost, which belongs in the trade-off when you decide where to place one.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
A capability-oriented pattern
OWASP also describes a design that separates duties: a privileged planner that does not inspect risky documents, a quarantined parser with no tools, and a custom interpreter that tracks data flow and enforces policy. OWASP cautions that this example has limitations and is not a supported security component. Treat it as a design to test against, not a drop-in fix, and do not treat any single architecture as a universal answer.
Published benchmarks as context
The USENIX Security 2024 paper “Formalizing and Benchmarking Prompt Injection Attacks and Defenses” evaluates five prompt-injection attacks and ten defenses across ten LLMs and seven tasks, and it provides a public platform for running such evaluations (USENIX Security 2024, Formalizing and Benchmarking Prompt Injection Attacks and Defenses). These are the study’s design counts. They are not a universal ranking of defenses, and they do not show how any particular application’s defenses perform.
A safe practice target before your own application
If you want to practice attack and evaluation technique without touching a real system, OWASP’s Basileak project is an intentionally vulnerable Falcon 7B fine-tune and a CTF sparring target for prompt-injection training, red-team education, and research (OWASP Basileak project). OWASP says it should not be deployed in production or used with real users, real data, or real credentials. Use it to learn the technique, and keep your evaluation pointed at a sandboxed copy of your own application.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
