Build the smallest, most isolated environment that can answer your defensive question, and assume every AI-generated action is untrusted until an operator, a policy gate, and a reset procedure have contained it. Do not route experimental traffic or commands to live operational technology (OT), and do not present a simulation as production-equivalent without evidence.
A sound evaluation combines synthetic or simulated process assets, strict network separation, narrowly scoped scenarios, complete telemetry, human approval for consequential actions, explicit stop conditions, and a tested recovery path. Increase realism—from software simulation to hardware-in-the-loop or a physical installation—only when the question requires it and the additional engineering controls are in place.
As an Amazon Associate I earn from qualifying purchases.
What a safe AI-driven OT test environment must accomplish
OT includes programmable systems and devices that interact with the physical environment. A controller instruction, altered setpoint, or changed logic can therefore affect safety, reliability, availability, the environment, and production. The purpose of the lab determines the fidelity you need: a test of alert routing may need realistic logs and network protocols, while a study of process response may require a high-fidelity process model or hardware-in-the-loop.
The official guidance available today is broad OT security guidance, not a dedicated standard for autonomous AI attack agents. NIST SP 800-82 Rev. 3, published in September 2023, remains the final revision identified by NIST. NIST lists SP 800-82 Rev. 4 as an initial public draft published September 21, 2026, with comments due November 30, 2026. The draft broadens sector coverage, aligns the guide with NIST Cybersecurity Framework 2.0, and expands treatment of asset management, monitoring and detection, and security architecture. Treat Rev. 4 as a draft, not as a final requirement.
#1 Best Overall
Use the guidance as context for safety, reliability, and performance constraints. The controls below are a practical design framework synthesized for AI-agent evaluations; they are not a verbatim NIST checklist.
Start with a defensive question and a written boundary
Write the evaluation objective before selecting tools or models. A useful objective states what decision the exercise will inform, which architecture and process are represented, what behavior is in scope, what evidence will be collected, and what would make the exercise unsafe.
Define the question
- Detection: Can the SOC identify and triage a sequence of suspicious OT events?
- Reporting: Can an AI system turn approved telemetry into an accurate incident report without inventing facts?
- Control validation: Does a proposed segmentation, allow-list, monitoring rule, or backup procedure work in a representative environment?
- Process response: How does a modeled process react when a bounded, reversible condition changes?
- Human oversight: Can an operator understand, approve, reject, and stop every consequential action proposed by an agent?
Record what is out of scope
List excluded assets, protocols, credentials, networks, physical interfaces, and actions. State explicitly that the exercise cannot reach production control networks, safety systems, remote vendor connections, or shared corporate services unless each connection is separately approved, isolated, monitored, and required by the question. In most evaluations, the safest answer is to use no production connection at all.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Set success and stop criteria
Define measurable evidence before the run begins: alerts generated, events retained, report fields completed, operator approvals recorded, recovery time, and discrepancies between the agent’s claims and observed facts. Define immediate stop conditions such as an attempted route outside the lab, an unapproved protocol, a command aimed at a real address, loss of logging, unexpected process-model behavior, or an operator’s safety concern. Give a named person authority to stop the exercise without seeking additional approval.
Choose the least realistic environment that can answer the question
There is no published head-to-head result establishing that one OT testbed type is universally safest or most useful for autonomous AI testing. Compare candidate environments against isolation, process and protocol fidelity, repeatability, reset and recovery, observable telemetry, scale, configuration effort, and the defensive question you defined.
| Environment | Use when | Isolation and physical effect | What to verify |
|---|---|---|---|
| Software-only simulation | The question concerns agent reasoning, workflow, detection logic, or reporting that does not require a physical response. | With no real control hardware or process connection, it can minimize direct physical effect; verify that no route, credential, or cloud integration leaves the lab. | Whether the simulator emits the protocols, timing, faults, and logs needed by the evaluation. |
| Simulated process with realistic interfaces | You need representative Modbus, OPC UA, historian, HMI, or network behavior while keeping the process virtual. | Use synthetic servers and models; verify that every interface terminates inside the lab and that reset returns the model to a known state. | Protocol fidelity, timing, data quality, alarm behavior, and whether the model represents the process decisions relevant to the question. |
| Hardware-in-the-loop | The question depends on controller firmware, I/O timing, engineering-workstation behavior, or other hardware characteristics that software cannot reproduce. | Use only dedicated equipment in a physically and logically isolated rig; verify that outputs cannot drive real actuators or shared plant systems. | Electrical and network interlocks, safe output states, deterministic reset, spare hardware, and an independent emergency stop. |
| Physical test installation | A specific engineering or process-response question cannot be answered with a model or HIL setup. | Assume the greatest potential consequence. Use nonproduction equipment, segregated utilities and networks, supervised operation, and documented safe states. | Physical safeguards, maintenance responsibility, environmental controls, recovery inventory, and evidence that the installation cannot affect a live process. |
Move to a more realistic tier only when the lower tier leaves an identified evidence gap. Document what the chosen tier reproduces and what it does not; realism is not the same as validity.
Build the isolation architecture before loading an AI agent
Separate the lab from live operations
Use dedicated hosts, switches, storage, identity, and management paths where feasible. Place the test environment behind explicit deny-by-default boundaries. Do not rely on a VLAN label, a virtual machine snapshot, or an agent prompt as the only isolation control. Remove routes to production and prevent accidental bridging through dual-homed laptops, shared jump hosts, remote-support software, synchronization services, or unrestricted Internet access.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Use synthetic assets and identities
Create fictional IP addresses, hostnames, users, certificates, controller projects, tags, recipes, and process values. Seed the environment with data that cannot be mistaken for a live facility’s identifiers. If real configuration or log samples are needed, sanitize them, remove secrets and operational details, and store the originals outside the agent’s reach.
Control every data flow
List inbound and outbound flows for the model, OT protocols, telemetry collectors, orchestration system, update mechanism, and analyst workstation. Permit only the ports and directions required for the stated test. Prefer one-way export of telemetry to an analysis zone over bidirectional access. If an agent needs an external language model or service, treat that connection as a separate risk: minimize the data, use synthetic content where possible, and ensure the service cannot issue commands back into the lab.
Make the safe state physical and logical
For HIL or physical rigs, define safe output values, remove or disconnect actuators that could cause harm, and add independent interlocks. For a simulation, define a model-safe state and a kill mechanism that does not depend on the agent, the model under test, or the same host. Test the stop mechanism before every exercise.
Constrain the AI agent’s authority
Model capability is not a safety control. Give the agent the minimum access needed for the question, and separate observation from action.
Recommended Free Tools
Use a capability ladder
- Read-only observation: The agent receives synthetic logs, packet captures, process values, or screenshots and can produce hypotheses or reports.
- Proposed action: The agent may generate a command, configuration change, or test step as text, but an operator or policy engine must approve it before execution.
- Sandbox execution: Approved actions run against disposable virtual assets with bounded privileges and automatic timeouts.
- Controlled process interaction: Only after lower tiers answer the question should an agent interact with dedicated controllers or a process model, with allow-listed functions, rate limits, and independent interlocks.
Do not skip tiers merely because an agent can technically reach a service. Separate the account that reads telemetry from any account that can change a model. Use short-lived credentials, narrowly scoped roles, and automatic revocation at the end of a run.
Put policy gates outside the agent
Enforce address, protocol, function, value, rate, time, and sequence limits in a control layer the agent cannot modify. Require explicit approval for writes, logic downloads, firmware operations, safety-related changes, credential use, or any action that could make a process model unstable. Log the original request, the agent’s proposed action, the policy decision, the approver, the executed result, and the resulting process state.
Defend against prompt and data manipulation
Assume that a synthetic device description, log entry, document, or network message may contain instructions intended to redirect the agent. Keep untrusted content separate from system policy. Tell the agent which fields are evidence and which are instructions, reject tool calls that do not match the scenario allow-list, and require the agent to cite the telemetry supporting each conclusion. Test refusal, escalation, and uncertainty behavior rather than rewarding aggressive execution.
Design threat-informed scenarios, not generic attack recipes
Use MITRE ATT&CK for ICS to describe adversary behavior and organize scenarios, but never treat a technique catalog as permission to test live equipment. MITRE warns that “Logic executing in ICS has a direct effect on the physical world.” Its guidance also notes that improper execution can have serious human-safety, environmental, and economic consequences.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsMITRE’s threat-informed failure-scenario methodology maps adversary techniques to an organization’s particular architecture and processes. Use that approach to create a bounded story: an initial condition, the assets represented, the behavior under evaluation, expected defensive observations, operator decisions, and a recovery point.
Example scenario record
- Objective: Determine whether monitoring and operator review detect an unauthorized change in a simulated water-treatment sequence.
- Represented assets: Synthetic engineering workstation, controller, HMI, historian, network sensor, and process model.
- Allowed behavior: The agent may inspect generated telemetry and propose a bounded test change; a human must approve execution.
- Expected evidence: Authentication events, protocol requests, process-value changes, alarms, policy decisions, and report citations.
- Abort conditions: Any request outside the synthetic address space, loss of telemetry, uncontrolled value change, or failed reset.
- Recovery point: Snapshot or scripted rebuild that restores identities, projects, data, and model state.
Instrument the environment for evidence and accountability
Collect enough synchronized evidence to reconstruct what the agent saw, inferred, proposed, and did. At minimum, capture agent prompts and outputs, tool calls, policy decisions, approvals, network metadata, OT protocol transactions, controller or simulator state, alarms, process values, host events, and operator actions. Use a common time source inside the lab and record clock health; otherwise, event ordering can be ambiguous.
Protect logs from alteration by the agent and by the systems being tested. Export copies to a separate analysis zone, restrict deletion, and preserve the exact model, prompt, policy, software version, configuration, and scenario seed used for each run. Record false positives, missed detections, unsafe proposals, hallucinated observations, and the time required for an operator to understand and stop the behavior.
Reset, recover, and prove the lab can be stopped
Prepare disposable states
Keep versioned images, controller projects, simulator seeds, synthetic databases, certificates, and configuration manifests. A reset should rebuild the environment rather than depend on an operator remembering every change. Separate baseline snapshots from experiment artifacts and verify their integrity before use.
Run recovery drills
Before evaluating the agent, deliberately trigger the stop path. Confirm that the emergency stop works when the orchestration server, agent, or network sensor is unavailable. Restore the baseline, verify asset identities and process values, and compare a post-reset checklist with the known-good state.
Best Value
Define evidence of recovery
Recovery is complete only when the cause of the stop is recorded, unauthorized changes are identified, credentials are rotated where necessary, logs are preserved, and an independent reviewer confirms that the environment is ready for another run.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Validate security products and behaviors before production use
Representative testing is a deployment gate, not a demonstration after installation. MITRE mitigation M0949 states: “Within industrial control environments, antivirus/antimalware installations should be limited to assets that are not involved in critical or real-time operations. To minimize the impact to system availability, all products should first be validated within a representative test environment before deployment to production systems.”
Apply the same discipline to detection rules, protocol inspection, endpoint agents, backup restores, segmentation changes, and AI-assisted workflows. A product that works in an IT sandbox may change timing, consume resources, or interfere with an OT workload. Test the specific versions, policies, and configurations intended for deployment, and retain the evidence supporting the release decision.
What published examples demonstrate—and what they do not
Simulation-based cyber ranges
The ICSrange paper by Vincenzo Giuliano and Valerio Formicola (2019) describes a simulation-based cyber range that virtualizes an enterprise network connected to ICS and simulated processes. It reports a multi-stage exercise involving a simulated water-tank environment. This is a research example of a range architecture, not evidence of a current commercial offering or a universal safe design.
LLM-assisted reporting
The 2026 ICSSPulse preprint by Michail Takaronis, Athanasia Kollarou, Vyron Kampourakis, Vasileios Gkioulos, and Sokratis Katsikas describes an LLM-assisted reporting module evaluated with synthetic Modbus servers, a Factory I/O water-treatment scenario, and a custom OPC UA production-line model. The authors identify software-only, unauthenticated testbeds as constraints on performance and experimental fidelity and characterize the system as suited to small- and medium-scale laboratory environments. This supports a reporting use case; it does not show that autonomous AI agents can safely conduct attack evaluations against realistic OT systems.
Common design failures and corrective actions
- “The lab is isolated because it is virtual.” Corrective action: remove routes, block shared services, inspect hypervisor and host interfaces, and test that a synthetic command cannot reach a real address.
- “A prompt tells the agent not to cause harm.” Corrective action: enforce external policy gates, least privilege, allow-lists, approvals, rate limits, and independent stop controls.
- “A generic ATT&CK technique is the scenario.” Corrective action: map the behavior to the represented architecture, process, telemetry, operator decisions, and recovery state.
- “More realism is always better.” Corrective action: identify the unanswered question that requires additional fidelity and document the new hazards and controls before adding hardware or process connections.
- “A successful run proves production safety.” Corrective action: state exactly what was represented, repeat tests across relevant configurations, and treat production deployment as a separate engineering and change-control decision.
- “Reset means restoring a snapshot.” Corrective action: verify identities, projects, credentials, logs, process state, and interlocks with an independent checklist after every reset.
- “The report is the evidence.” Corrective action: preserve raw telemetry, tool calls, policy decisions, approvals, and timestamps so another reviewer can reproduce the conclusion.
A practical build sequence
- Write the defensive question, represented process, scope, exclusions, success measures, and stop conditions.
- Select the lowest-fidelity environment that can produce the required evidence.
- Inventory every asset, interface, identity, data flow, and physical dependency in the proposed lab.
- Create synthetic assets and credentials; sanitize any retained examples.
- Implement deny-by-default segmentation and verify that no path reaches live operations.
- Build the process model or dedicated hardware, then define safe states, interlocks, and reset artifacts.
- Install telemetry collection and an independent evidence store before connecting the agent.
- Start the agent in read-only mode, test refusal and escalation, and compare its output with ground truth.
- Add proposed-action and then bounded-execution stages only when the prior stage meets its acceptance criteria.
- Run the stop and recovery drill, preserve all artifacts, review deviations, and approve the next level of realism—or end the exercise.
Bottom line for practitioners
A safe AI-driven OT evaluation is a controlled experiment, not an unfiltered attack playground. Keep experimental traffic and commands away from live assets, make the agent’s authority smaller than the environment’s safeguards, tie scenarios to a specific architecture and process, and increase fidelity only when the evidence requires it. Simulation and LLM-assisted research provide useful starting points, but no published example establishes a universal topology or validated safeguards for autonomous AI attack agents.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Free tools Windows power users keep installed
One-click scans. No signup required.
