An LLM stub replaces a model call with a predetermined or request-aware response, so you can test how your agent application handles that response without contacting a model provider. Use it to make orchestration tests repeatable: verify tool selection and authorization, guardrails, handoffs, retries, state changes, and failure handling. A passing stub test does not show that a real model will make the same safe choice or resist a new attack. Pair scripted tests with model-backed evaluations and adapter tests that exercise the provider connection.
What an LLM stub can—and cannot—test
An agent typically receives model output, decides whether to invoke a tool or hand off work, applies application policies, and then handles the resulting tool output or error. A stub lets a test control the model-output step while the application runs its normal orchestration path. That makes it possible to reproduce a particular tool request, refusal, malformed response, or sequence of turns on demand.
As an Amazon Associate I earn from qualifying purchases.
| Test layer | What it can establish | What it cannot establish by itself |
|---|---|---|
| Scripted-model unit or workflow test | How application code routes a response, validates tool arguments, applies authorization and approval rules, handles guardrails, retries, handoffs, sessions, and state transitions. | Whether a live model will produce that response, follow instructions reliably, choose the right tool, or resist an unfamiliar attack. |
| Adapter test with a controlled or mocked transport | Whether the real provider adapter builds and parses requests as expected, including serialization, headers, defaults, and response handling. | Actual provider authentication, network behavior, or end-to-end execution unless those are explicitly included in the test setup. |
| Provider or sandbox integration test | Behavior of the real connection or isolated tool execution in the environment actually exercised. | Broad safety or security across all models, tasks, attack strategies, configurations, or production conditions. |
| Model-backed evaluation or red-team test | Model-dependent outcomes for the tested model, configuration, tasks, attack cases, and attempts. | A guarantee that the system is safe in untested circumstances or will remain safe after a change. |
Keep these results separate in reports. A deterministic orchestration test is valuable evidence about your code, not a substitute for evaluating a probabilistic model or testing the provider boundary.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →How to build a deterministic agent test
Prefer the model abstraction or test-double mechanism supported by your framework over patching unrelated internals. OpenAI’s current Agents SDK testing guides for Python and JavaScript describe deterministic, provider-neutral test utilities, including ScriptedModel-style doubles; these tests can exercise orchestration without making model-provider requests. LangChain Core’s v1.6.2 reference documents fake chat-model options including FakeMessagesListChatModel, FakeListChatModel, and GenericFakeChatModel. Their APIs and behavior are framework- and version-specific, so follow the documentation for the package you use rather than assuming one fake works identically across languages or releases.
#1 Best Overall
- Arrange a scripted response sequence. Define the exact assistant message or tool-call response the scenario needs. For a tool workflow, include the model response that requests the tool and the later response expected after the tool result is returned.
- Run the production entry point. Call the same application-level agent workflow used in production, with only the model implementation and effectful dependencies replaced by test doubles.
- Instrument the boundary. Record normalized model input, requested tool and arguments, validation result, authorization decision, approval state, tool result, and any dummy-state mutation. Ensure a prohibited action cannot reach a real effectful implementation.
- Assert the complete outcome. Check the response, the calls and arguments, policy decisions, state changes, and errors. Also check that the scripted model consumed the expected steps; an unused or unexpectedly exhausted step can reveal a changed control flow.
- Keep provider behavior in a separate test. Exercise the actual adapter with a mocked or controlled HTTP transport when checking request serialization, headers, defaults, or provider response parsing. Use a sandbox or provider integration test for behavior that depends on a real connection or execution environment.
A deliberately simple scenario might script a request to a read-only lookup tool, return synthetic data from an instrumented tool, and then script the final answer. The test should assert not just that the answer looks right, but that the agent requested the intended tool with validated arguments, authorization allowed that specific action, and only the fake tool’s state changed. Use synthetic credentials and marker data; do not place live secrets or customer records in fixtures.
Which security cases should the test suite cover?
Start with an abuse-case matrix tailored to your agent’s supported tasks, permissions, and input channels. OWASP’s AI Agent Security Cheat Sheet calls for structured security testing, while its LLM Prompt Injection Prevention Cheat Sheet describes example attacks as illustrative smoke tests—not a representative security benchmark. Include benign controls as well as abuse cases: a system that refuses every request should not pass as a useful, secure agent.
| Case | Where to place the test input | What to observe |
|---|---|---|
| Direct prompt override | User message or other direct instruction channel. | Whether policy is maintained and whether an unauthorized tool request is blocked before execution. |
| Indirect prompt injection | The external content channel under test: for example, a retrieved document, web result, message, or tool output the agent actually ingests. | Whether untrusted content is treated according to policy; record requested actions, arguments, authorization results, and dummy-state changes. A payload only placed in the user message does not test the same trust boundary. |
| Unauthorized tool use or privilege escalation | A scripted response requesting a tool or capability the test identity is not allowed to use. | That ordinary application authorization denies the request and the effectful implementation is never reached. |
| Malformed arguments or denied approval | A tool call with invalid arguments, or a workflow in which required human approval is denied. | Validation and approval outcomes, including confirmation that denial prevents the action rather than merely changing the final wording. |
| Memory poisoning or sensitive-data exfiltration | Untrusted content that attempts to alter retained memory or obtain synthetic sensitive marker data. | Whether memory writes are controlled and protected data is withheld from tools or outputs outside the permitted policy. |
| Recursive tool use, retries, or resource exhaustion | A scripted sequence that repeatedly requests tools, fails, or would exceed configured retry, token, or cost limits. | Bounded attempts, timeout and error handling, and expected circuit-breaker behavior; verify the test cannot create unbounded work. |
| Multi-agent boundary violation | A handoff or message that asks another agent to exceed its role or permission boundary. | Handoff target, delegated context, downstream authorization, and whether the receiving agent can cause a prohibited effect. |
For every case, write down the threat, input surface, intended policy, safe synthetic context, and observable pass condition before running it. Instrument fake tools to record attempted actions and arguments while preventing access to production systems. Inspect the action trail, not only the final answer: a later refusal does not undo a tool action that already occurred.
Keep prompts and expected denials version controlled, review test changes alongside agent behavior changes, and retain regressions for known failures. If the test environment emits traces, disable tracing or capture it locally so fixtures and test activity are not unintentionally exported.
Rank #3
What stubbing does not prove about prompt injection
A scripted model has been told by the test author what response to emit. The test establishes how surrounding code behaves when that response arrives; it does not establish how a deployed model will react to a new malicious instruction, select a safe action, or follow policy consistently. Test those model-dependent claims with evaluations and red-team cases against the actual supported system configuration.
Model outputs vary between attempts. In its January 17, 2025 article on agent hijacking evaluations, NIST’s Center for AI Standards and Innovation recommends adaptive evaluation and task-specific analysis in addition to aggregate results, and discusses the need for multiple attempts. In one held-out task evaluation described in that article, the strongest new attack raised measured attack success from 11% for the strongest baseline to 81%. Those figures belong to that evaluation’s models, tasks, attack methods, and setup; they are not a general agent failure rate.
Evaluation environments can help make attack and defense comparisons more structured, but they are not production certifications. The AgentDojo authors’ June 19, 2024 paper describes an extensible environment with 97 realistic tasks and 629 security test cases in that research release. The paper also notes that state-of-the-art models can fail ordinary tasks without an attack. Treat benchmark performance as evidence about the tested setup, not a guarantee for your own tools, permissions, or input channels.
- Check whether the evaluation tasks and tools resemble the work your agent is meant to do.
- Confirm that direct and indirect attack channels are represented, and whether attacks adapt to the model’s responses.
- Record attempts per case and report task-specific outcomes as well as aggregate scores.
- Review whether traces and audit evidence are sufficient to explain actions, not just final answers.
How to turn test results into governance evidence
Translate broad security expectations into verifiable requirements, then exercise the application’s real trust boundaries with operational abuse cases. OWASP LLMSVS v2.0, published in 2026, organizes verification into eight groups (V1–V8), covering areas that include secure configuration and maintenance, model lifecycle, model memory and storage, secure LLM integration, agents and plugins, dependencies, and monitoring. It is a verification framework, not a certification: consulting its checklist does not by itself establish that a system is secure.
For each release, retain an evidence record that makes the test reproducible and the decision reviewable:
- Agent version and relevant model provider, model, and configuration identifier.
- Tool permissions and policy, retrieval and memory configuration, and relevant approval settings.
- Test-case and fixture identifiers, expected outcomes, observed results, and the environment used.
- Tool calls and arguments, authorization and approval outcomes, timeouts, retries, and circuit-breaker behavior.
- Failures, remediation, regression coverage, accepted residual risk, and any compensating controls.
Rerun relevant tests before release and after material changes to prompts, tools, memory, retrieval, policies, or model providers. Preserve earlier failures as regression cases, and review changes that weaken or remove a security test with the same care as changes to agent behavior.
NIST ITL’s ongoing 2026 agentic AI evaluation-probe project offers a useful traceability pattern: map claims or decisions to source evidence and assess whether the evidence supports the claim (faithfulness), captures the source’s message (completeness), and meets the claim’s evidentiary burden (sufficiency). That project concerns evaluation probes and grounding; use the pattern to improve audit trails, not as a complete security-governance standard.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
