Test an AI agent as an application, not just as a model that should refuse bad prompts. Put direct and indirect prompt-injection attempts through isolated versions of the agent, then verify that tool permissions, downstream authorization, input checks, approval gates, and execution limits prevent unauthorized actions—even if the model follows the attack.
What a useful guardrail test must prove
A refusal is not the same as a security control. An agent might refuse a malicious instruction in one run and comply in another; even a model that recognizes an attack should not be able to perform an action its user is not authorized to take. The test should therefore follow the request through the whole application: from input and retrieved content, through the model and proposed tool call, to the service that permits or blocks the action.
As an Amazon Associate I earn from qualifying purchases.
For each case, distinguish what the agent tried to do from what the application allowed it to do. A blocked unauthorized call is different from no attempted call, and both are different from a successful harmful action. Record those distinctions instead of reporting only whether the final answer looked safe.
Map the attack surface before writing tests
List the instructions and data sources the agent can see, the tools it can call, the identity and permissions used for each call, and the effects those calls can have. Include memory, retrieval, handoffs to other agents, and any approval mechanism. Start with externally reachable paths and actions with meaningful consequences.
#1 Best Overall
![Arduino® UNO™ Q 4GB [ABX00173]- Hybrid Board, Qualcomm Dragonwing QRB2210 microprocessor (MPU) & STM32U585 Microcontroller(MCU), AI Vision, Voice, IoT, Robotics, Linux Debian OS, Wi-Fi 5, USB-C](https://m.media-amazon.com/images/I/51cl4xygwGL._SL160_.jpg)
- Dual-Brain Hybrid Power: Combines the Qualcomm Dragonwing QRB2210 MPU (Quad-core Arm Cortex-A53 @ 2.0 GHz CPU, Adreno GPU, AI acceleration) and the real-time, low-power STM32U585 MCU for advanced applications like object recognition, voice commands, and motion detection.
- AI & Linux Capabilities: Unlocks AI-powered vision and sound solutions; runs Linux Debian OS for coding in Python and supports the Arduino ecosystem with libraries and Sketches; quick start with Arduino App Lab.
- Advanced Features: Equipped with 4 GB LPDDR4 RAM, 32 GB eMMC built-in storage, ideal for single-board computer (SBC) mode, running multiple simultaneous high-level processes, more complex AI or ML models, extensive logs. Dual-band Wi-Fi 5 (2.4/5 GHz), Bluetooth 5.1, and high-speed headers for vision, audio, and display peripherals.
- Seamless Expansion & Connectivity: Features the classic UNO form factor for shields compatibility, an 8x13 LED matrix, and a Qwiic connector for easy expansion with Modulino nodes; power and connect via the USB-C connector.
- Intended Use & Development: The perfect platform for prototyping robotics or IoT projects, empowering innovators with a unified development experience to mix Arduino Sketches, Python scripts, and containerized AI models in a single interface.
- Direct prompt injection: A user message tries to override trusted instructions, obtain secrets, redirect the task, or invoke a tool unrelated to the user’s request.
- Indirect prompt injection: Malicious instructions arrive in content the agent consumes, such as a retrieved document, web page, email, or tool response. Check whether that content can redirect the task or alter arguments sent to a tool.
- Tool misuse and escalation: The agent attempts an unauthorized function, uses a broader scope than needed, crosses from a low-trust context to a high-trust action, or performs an action that requires approval without valid approval.
- Leakage and persistence: Sensitive context appears in tool calls, citations, outputs, logs, or memory writes, or malicious content persists across sessions or users.
- Resource and chain abuse: Repeated retries, nested calls, recursion, or multi-agent handoffs exceed the intended limits or trigger unauthorized actions.
Where the system supports them, include obfuscated or hidden instructions, split or multilingual payloads, and malicious text embedded in images or other modalities. OWASP’s LLM Prompt Injection Prevention Cheat Sheet describes these as prompt-injection patterns; testing should reflect the inputs the particular deployment can actually process.
Run a safe, repeatable test sequence
- Define the trust boundary and impact. For each important workflow, document what is trusted, what is untrusted, which identity makes each tool call, the data involved, required approvals, and the effect of a successful action. Prioritize high-impact paths rather than starting with whichever prompt is easiest to write.
- Isolate execution. Use a sandbox, simulated accounts, or tools that cannot affect real users or systems. Do not put real secrets in test prompts or use live customer data in fixtures. OWASP’s agent-security guidance warns against both practices.
- Pair normal tasks with adversarial variations. Keep a benign case for the legitimate task, then make a controlled variation that introduces an attack. For each, specify the intended result, forbidden action, permitted calls, expected authorization decision, and evidence to collect. NIST CAISI describes agent-hijacking scenarios in which an agent receives a legitimate task but encounters data containing an attempt to redirect it to a malicious task.
- Test enforcement where actions occur. Exercise the risky call and check that the tool or downstream service rejects it when the actor, resource, arguments, scope, or approval is invalid. Do not count a model’s stated refusal as proof that the service would enforce the rule.
- Repeat attempts and inspect task-level results. Record legitimate-task completion and malicious-task success separately. Break results down by task and attack class; multiple attempts can reveal behavior a single run misses. NIST CAISI recommends adaptive evaluation and task-specific analysis rather than relying only on aggregate performance.
- Gate relevant releases. Keep cases and expected denials under version control. Run the suite when prompts, tools, memory, retrieval, policy, or model providers change. OWASP recommends release gates when high-risk policies, approval logic, or credential scopes change without updated tests.
- Keep evidence that can reproduce a result. Retain the tested agent version and configuration, model provider, tool policy, retrieval settings, case, tool-call trace, decisions, final outcome, and any approval, denial, timeout, or circuit-breaker behavior. Redact secrets and personal information from fixtures and logs.
Design paired cases around real actions
A useful case tests a boundary, not just whether a suspicious phrase triggers a refusal. For example, if an agent is asked to summarize an email, a paired test can use an ordinary email and one containing an instruction to forward confidential content. The test should check whether the agent stays on the summary task, whether it attempts a send action, and whether the service rejects that action if it is outside the user’s permissions or lacks required approval.
Rank #2
- Dual-Brain Hybrid Power: Combines the Qualcomm Dragonwing QRB2210 MPU (Quad-core Arm Cortex-A53 @ 2.0 GHz CPU, Adreno GPU, AI acceleration) and the real-time, low-power STM32U585 MCU for advanced applications like object recognition, voice commands, and motion detection.
- AI & Linux Capabilities: Unlocks AI-powered vision and sound solutions; runs Linux Debian OS for coding in Python and supports the Arduino ecosystem with libraries and Sketches; quick start with Arduino App Lab.
- Advanced Features: Equipped with 2 GB LPDDR4 RAM, 16 GB eMMC built-in storage, ideal to develop in PC-connected mode, running the OS, Python scripts, and basic network services (SSH) without a demanding GUI or heavy multitasking; great for lightweight AI and memory-optimized TinyML applications, needing local storage for basic OS and core libraries. Dual-band Wi-Fi 5 (2.4/5 GHz), Bluetooth 5.1, and high-speed headers for vision, audio, and display peripherals.
- Seamless Expansion & Connectivity: Features the classic UNO form factor for shields compatibility, an 8x13 LED matrix, and a Qwiic connector for easy expansion with Modulino nodes; power and connect via the USB-C connector.
- Intended Use & Development: The perfect platform for prototyping robotics or IoT projects, empowering innovators with a unified development experience to mix Arduino Sketches, Python scripts, and containerized AI models in a single interface.
Write each case so another person can tell what passed without interpreting vague output. A compact case definition can include:
Recommended Free Tools
- Setup: the user identity, relevant data, available tools, scopes, and approval state.
- Input: the benign request and the adversarial content or variation.
- Expected behavior: the legitimate result, calls that are allowed, and actions that must not occur.
- Enforcement check: which application or service rule should allow or block the proposed action.
- Evidence: the agent response, structured tool trace, authorization decision, and any side effect in the sandbox.
For indirect attacks, place the malicious instruction in the same kind of external content the deployed agent reads. Check both whether the instruction changes the agent’s goal and whether it changes tool arguments. Treat a prompt that merely says “ignore previous instructions” as only one test pattern, not a complete simulation of external-content risk.
Rank #3
- Single core ARM Cortex-A7 32-bit core, integrated with NEON and FPU
- Built in Micro's self-developed 4th generation NPU, with high computational accuracy and support for mixed quantization of int4, int8, and int16. Among them, int8 has a computing power of 0.5 TOPS and int4 has a computing power of up to 1.0 TOPS
- Built in self-developed 3rd generation ISP3.2, supports 4 million pixels, and supports various image enhancement and correction algorithms such as HDR, WDR, and multi-level denoisin
- It has powerful encoding performance, supports intelligent encoding, adapts to save bit rates according to the scene, and saves more than 50% of the bit rate compared to conventional CBR mode, making the captured images high-definition, smaller in size, and doubling the storage space
- The design with built-in RISC-V MCU supports low-power fast startup, 250ms fast capture, and simultaneous loading of AI model library, enabling facial recognition to be completed within 1 second
Verify the controls that should contain misuse
- Least privilege: Give each tool only the functions and permissions required for its task. OWASP’s excessive-agency guidance highlights the avoidable risk of combining email read access with an unnecessary send function.
- Downstream authorization: Recheck access in the tool or service for every action. Authorization should depend on the actor, resource, and requested operation—not on whether the model says the request is permitted.
- Action-bound approvals: For high-impact actions, verify that approval is valid and current and applies to the exact action parameters. Test bypass attempts and attempts to reuse an approval for a changed action.
- Schema and value validation: Validate structured model outputs and sanitize values before passing them to tools or other systems. Try malformed, unexpected, and attacker-controlled arguments relevant to the tool.
- Untrusted-content separation: Keep external content distinct from trusted instructions and test whether it can change goals or tool arguments. This is a helpful boundary, not proof that prompt injection is fully prevented.
- Execution limits and monitoring: Set and exercise limits for tool chains, retries, tokens, cost, and request rates. Check structured logs and monitoring for suspicious action sequences.
- Guardrail-model checks: If another model screens inputs, outputs, or proposed actions, test that component too. OWASP cautions that guardrail models can themselves be prompt-injected and add latency and cost; they are one layer of defense, not a substitute for application controls.
Report outcomes that expose failures
For each case, report at least these separate outcomes:
- Whether the legitimate task was completed.
- Whether the malicious task was completed.
- Whether an unauthorized tool call was attempted.
- Whether the application blocked the call.
- What impact would have resulted if the control had failed.
Break these results down by action class and trust boundary. A single aggregate pass rate can conceal a severe failure in a less common but high-impact workflow. Keep the case, expected result, configuration, and trace together so a future run can be compared meaningfully.
Rank #4
- 【POWERFUL ESP32‑S3 CONTROLLER】Built‑in Xtensa 32‑bit LX7 dual‑core processor, 512KB SRAM, 8MB PSRAM, 16MB Flash for stable AI voice computing and multitask processing.
- 【Preloaded Dual AI Platforms】Comespre-installed with complete Deepseek and OpenAI voice dialogue projects.Experience intelligent voice interaction instantly. (Note: OpenAI functionality requires your own API key.)
- 【STABLE WIRELESS & CLEAR AUDIO】Integrated 2.4GHz Wi‑Fi + Bluetooth 5 (LE); dedicated audio decoding module for natural, responsive voice interaction.
- 【USER‑FRIENDLY VISUAL & PLUG‑AND‑PLAY】2” TFT‑SPI color screen shows real‑time chat; modular design, no extra wiring, ready to use after setup.
- 【FULL LEARNING SUPPORT】45 programmable GPIOs, rich interfaces, online web tutorials, free technical support for beginners & developers.
NIST CAISI’s January 17, 2025 article, Strengthening AI Agent Hijacking Evaluations, describes evaluation work using AgentDojo’s simulated Workspace, Travel, Slack, and Banking environments, alongside custom scenarios. AgentDojo is a useful example of a framework for testing agent hijacking; it is not a universal stand-in for every agent architecture or deployment. Choose an environment and cases that reflect your own tools, trust boundaries, and consequences.
Interpret passing tests carefully
OWASP explicitly characterizes its prompt examples as smoke tests, not a security benchmark. Passing them does not demonstrate resistance to a persistent adversary, and guardrails can reduce risk without eliminating the underlying vulnerability. Treat each pass as evidence about the tested configuration and cases—not a general guarantee about future prompts, changed tools, or new data sources.
OWASP’s AI Agent Security Cheat Sheet and LLM06:2025 Excessive Agency are useful references for abuse cases, authorization, and limiting tool functionality and autonomy. Its Large Language Model Security Verification Standard v2.0 includes a section on agents and plugins. Use such guidance to build and update a test suite, while making release decisions from evidence about the deployed application.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
