Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →You can add an adversarial scan to your pull-request checks in GitHub Actions and have it fail a PR when the agent’s behavior regresses, even when the code and unit tests pass. In one published example, a single prompt change let a support agent issue refunds from an order ID and amount without checking the order, and the scan turned the PR check red. The author reports that one scan cost about $0.26 in model calls for the attacker and judge roles, under a specific configuration described below.
What the change broke, and why ordinary tests missed it
The example comes from Ayan Pahwa’s case study for Humanbound, published September 23, 2026: Red-team AI agent pull requests in GitHub Actions. The change was a prompt edit. It told the support agent to issue a refund based on an order ID and an amount, without verifying that the order existed or belonged to the customer.
As an Amazon Associate I earn from qualifying purchases.
Nothing about that edit would show up in a conventional test suite. No function signature changed, no schema broke, and the text was valid. The author says the change passed ordinary checks. What changed was behavior: the agent would now act on a claim it had never verified. Prompts, tool descriptions, and policy files can alter what an agent does under hostile or unusual input without touching any code path a unit test exercises.
That is the gap adversarial testing fills. OpenAI describes red teaming as a complement to ordinary evaluations, not a replacement for them: OpenAI’s red-teaming guide states, “Red teaming uses adversarial test cases to help uncover unsafe, insecure, or policy-violating behavior before deployment.” Adversarial tests probe misuse, failure modes, and unexpected interactions that a fixed set of expected-input checks does not reach.
#1 Best Overall
- POWERFUL SECURITY KEY: The Security Key C NFC is the essential physical passkey for protecting your digital life from phishing attacks. It ensures only you can access your accounts.
- WORKS WITH 1000+ ACCOUNTS: Compatible with Google, Microsoft, and Apple. A single Security Key C NFC secures 100 of your favorite accounts, including email, password managers, and more.
- FAST & CONVENIENT LOGIN: Plug in your Security Key C NFC via USB-C and tap it, or tap it against your phone (NFC) to authenticate. No batteries, no internet connection, and no extra fees required.
- TRUSTED PASSKEY TECHNOLOGY: Uses the latest passkey standards (FIDO2/WebAuthn & FIDO U2F) but does not support One-Time Passwords. For complex needs, check out the YubiKey 5 Series.
- BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.
Wiring the scan into a pull-request gate
The case study’s setup has four parts: a path-filtered trigger on each PR, a single-turn scan that can block merges, a deeper multi-turn scan on a schedule, and a manual run before releases. The article’s workflow file is not reproduced here. Check its syntax and the current action versions against the author’s page and GitHub’s Actions documentation before copying anything.
Trigger on the behavior surface
The example runs when files that shape agent behavior change. The paths it lists are:
- Agent code
- Prompts
- Tool definitions and configuration
- Scope or policy files that define what the agent may do
- The workflow file itself
Including the workflow file means a change to the gate is itself gated. Treat the list as a starting point; your repository’s layout will differ.
Free tools Windows power users keep installed
One-click scans. No signup required.
Single-turn scan on every pull request
The PR job runs a single-turn scan: each adversarial prompt is one shot, with no follow-up. The job fails when the scan reports a high-severity finding. In the author’s demonstration run, the PR scan sent 432 attacks.
Rank #2
- POWERFUL SECURITY KEY: The Security Key NFC is the essential physical passkey for protecting your digital life from phishing attacks. It ensures only you can access your accounts.
- WORKS WITH 1000+ ACCOUNTS: Compatible with Google, Microsoft, and Apple. A single Security Key NFC secures 100 of your favorite accounts, including email, password managers, and more.
- FAST & CONVENIENT LOGIN: Plug in your Security Key NFC via USB-A and tap it, or tap it against your phone (NFC) to authenticate. No batteries, no internet connection, and no extra fees required.
- TRUSTED PASSKEY TECHNOLOGY: Uses the latest passkey standards (FIDO2/WebAuthn & FIDO U2F) but does not support One-Time Passwords. For complex needs, check out the YubiKey 5 Series.
- BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.
Multi-turn scan on a schedule
The optional scheduled job uses a deeper multi-turn mode, in which the attacker can carry on a longer conversation. This mode finds different kinds of failures, but each run takes longer and costs more. Scheduling it, rather than running it on every PR, is the trade-off the example makes.
Manual dispatch before release
A manual workflow_dispatch run supports pre-release scans, so a full pass can be started before a version ships without waiting for a PR.
Concurrency, timeouts, and metering
- Cancel superseded runs. When a newer push lands on the same PR, the older scan is cancelled so you do not pay for results nobody will read.
- Set job timeouts. A bounded job time caps runaway conversations and stuck calls.
- Meter tokens. The author counts attacker and judge tokens to estimate spend. Use your model provider’s usage data or an equivalent count so the figure is yours, not borrowed.
Results: SARIF and artifacts
- SARIF upload. The example uploads results in SARIF, a standard format for analysis findings, so reviewers can see them alongside other code-scanning output.
- Stored artifacts. Run artifacts can contain full adversarial transcripts. The author warns that on public repositories, artifacts can be downloaded by signed-in GitHub users. Decide what gets stored, for how long, and whether a public repository should store transcripts at all.
Reading a red build: establish a baseline first
The most useful detail in the case study is that its main branch was also red. A gate that counts every existing finding will fail PRs for problems they did not introduce. The author’s reported results were:
| Run (as reported by the author) | Volume | Pass | Fail |
|---|---|---|---|
| PR branch, single-turn | 432 attacks | 384 | 48 |
| Baseline branch, single-turn | 432 attacks | 393 | 39 |
| PR branch, multi-turn agentic | 97 conversations | 7 | 90 |
These numbers describe one demonstration agent and one configuration. They are not pass or fail rates for agents in general. Two readings are safe. The baseline already had 39 failures before the change, so the agent had findings unrelated to the edit. The PR single-turn run showed nine more failing attacks than the baseline run, but one run per branch cannot tell you whether that gap is signal or variation. The multi-turn row also shows that failure counts from different scan modes are not comparable, so a threshold set for one mode does not transfer to another.
Rank #3
A workable rollout follows from this:
- Run in report-only mode against the main branch. Record every finding, its severity, and the attack that produced it.
- Triage the existing findings. Assign an owner to each, and decide which are accepted risk and which must be fixed.
- Choose the blocking rule after triage. The example blocks on high-severity findings. Make sure the rule is one a PR can actually satisfy once the baseline is understood.
- When a PR goes red, read the specific finding. Reproduce the attack, then decide whether this PR introduced it. Compare against the baseline before assigning blame to the change.
What the 26 cents covers
The author reports about $0.26 per scan, measured across three runs in the case study. The figure covers the attacker and judge model calls, which used gpt-4o-mini, counted with the article’s token-metering method. The agent’s own model calls were not metered, and the author says scan cost varies with model choice and usage.
| Cost component | In the reported figure? | Basis |
|---|---|---|
| Attacker model calls | Yes | gpt-4o-mini in the example |
| Judge model calls | Yes | gpt-4o-mini in the example |
| The agent’s own model calls | No | Not metered in the case study |
| GitHub Actions runner time | Not stated | Not reported in the case study |
| Other attacker or judge models | Not stated | Cost depends on model choice, per the author |
| Human time to triage findings | Not stated | Not reported in the case study |
Scaled up, the arithmetic is simple: at the reported rate, 100 scans in the same configuration come to about $26 in attacker and judge calls, before the agent’s own model usage, runner time, and triage time. Your own bill will move with the models you choose and the length of your conversations.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Fork pull requests and secrets
The example skips pull requests from forks, because its workflow needs a secret that fork code should not reach. That protects the secret, but it also means outside contributions get no adversarial check in this setup. Decide this explicitly and document it, rather than letting the gap appear by default.
Recommended Free Tools
GitHub’s documentation for Copilot CLI in Actions makes the same concern concrete: “Workflows that run on pull request events from forks are at higher risk of prompt injection.” See GitHub Docs: About using Copilot CLI in GitHub Actions. That warning is specific to that product. Do not assume another Action handles fork events the same way; check its documentation.
Rank #4
- Ultra-Compact FIDO2 Security Key - Plug-and-stay or carry on a keychain. This USB-A hardware security key offers portable, always-on protection for desktop and mobile use. (Item Size: 0.75 X 0.74 IN x 0.25 IN)
- USB-A Hardware Key for All Devices - Works with USB-A ports on PC, Mac, Android, and other laptop/notebook device. Enables secure, cross-platform login with FIDO2.0 passkey support.
- FIDO Certified Security Key - Meets FIDO and FIDO2 standards. Works with Google, Microsoft, GitHub, Dropbox, and more. Please check service compatibility before purchase.
- Passwordless Login with Passkey - Supports passkey login via WebAuthn and CTAP2. Enjoy password-free sign-ins where supported. Not all websites or services currently support passkeys.
- Advanced Multi-Factor Authentication - Offers 200 FIDO2 passkey slots and 50 OATH-TOTP slots. Strong, flexible 2FA/MFA support across various apps and authentication platforms.
Permissions and other automation options
Give the scan job the narrowest token permissions and the fewest secrets it needs, and keep the secret out of any job that runs untrusted code. The case study does not describe its permission model in detail, so do not read the protections below into it.
GitHub’s separate product, Agentic Workflows, documents its own guardrails: read-only defaults, safe outputs, secret isolation, threat detection, and firewalled execution. These belong to that system. See GitHub Docs: About GitHub Agentic Workflows. Its tutorial on developing agentic workflows carries a public-preview notice, so check the page for current status before relying on it. It is a different product from the Humanbound Action used in the example.
What a green check does and does not establish
Adversarial scans probe for injection as well as policy violations. OpenAI defines the first this way: “Prompt injections occur when a third-party—not the user nor the AI—misleads the model by injecting malicious instructions into the conversation context.” (OpenAI: Understanding prompt injections.)
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsA passing scan means the attacks in that run did not produce a flagged failure. It does not mean the agent is secure. The attack set is finite, the judge is a model, and the agent’s behavior can change with the next prompt edit.
No independent study or official statistical series validating the case study’s scan counts or its cost estimate was found. Treat the 432-attack runs, the pass and fail counts, and the $0.26 figure as one author’s reported example, measured on one demonstration agent. Use them to plan your own measurement, not as a benchmark to compare against.
Humanbound supplies the Action in the example. The article is the source for the workflow details, run counts, and cost described above.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
