What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
AI systems have helped discover previously unknown vulnerabilities, but that does not show they can reliably avoid vulnerabilities when generating code. Finding, recognizing, fixing and preventing flaws are different tasks—and passing functional tests is not proof that software is secure.
How can AI find a zero-day but still write vulnerable code?
Finding a flaw in existing software and avoiding flaws while building new software ask different questions. In discovery, a system can inspect code, look for suspicious patterns, use fuzzing or test harnesses, and investigate a possible weakness. In code generation, it must meet functional requirements while also making safe choices across many details, including input validation, access control and data handling. A model that recognizes a flaw in one context may not consistently prevent the same class of flaw in new code.
As an Amazon Associate I earn from qualifying purchases.
“Zero-day” describes a vulnerability that was previously unknown, not a general level of skill. Anthropic reported in February 2026 that Claude Opus 4.6 found high-severity flaws in well-tested codebases, including some that had escaped detection for years or decades. That is a vendor-reported demonstration in particular projects, not evidence that AI systems generally find every kind of zero-day or can secure arbitrary software.
OpenAI’s 2026 GPT-6 Astra system card also reports that the system discovered and used previously unknown zero-day vulnerabilities in an internal evaluation. Separately, it reports a 100% “Cap Percent” result on ExploitBench, an evaluation involving 41 known V8 vulnerabilities. OpenAI cautions that possible contamination from historical vulnerabilities could inflate the benchmark result. That score measures performance on the ExploitBench task; it is not a zero-day discovery rate.
#1 Best Overall
Why are detection, repair and secure generation different skills?
A vulnerability detector is asked to judge code it can see. A code generator must produce an implementation that works and remains safe across relevant inputs and use cases. A repair system has another task: change existing code without breaking its intended behavior or leaving a related weakness behind. Success at one does not guarantee success at the others.
The authors of CoV-Eval, published by the Association for Computational Linguistics in July 2025, evaluate code completion, vulnerability detection, repair and classification as separate tasks. They report that many tested models could identify vulnerable code reasonably well, yet still tended to generate insecure code and had difficulty with particular vulnerability types and repairs. That difference helps explain why a system can be useful for security review without being a dependable source of secure code.
What do code-generation benchmarks show?
Several evaluations test generated code against security criteria, rather than assuming that a program is safe because it runs or passes its ordinary tests. Their results point to a persistent gap, but they use different tasks and denominators; the figures below should not be treated as a single ranking of models.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitches| Evaluation | What it tested | Reported finding | How to interpret it |
|---|---|---|---|
| SecRepoBench, Google Research, 2025 | Secure code completion in real-world repositories; 318 tasks across 27 C/C++ repositories and 15 CWE categories. | Code agents significantly outperformed standalone LLMs, but even state-of-the-art LLMs struggled to produce completions that were both correct and secure. | Repository context and an agent scaffold can help, but the benchmark does not show that either guarantees secure output. |
| BaxBench, Proceedings of Machine Learning Research, 2025 | 392 tasks generating backend applications, assessed for correctness and security. | OpenAI o1 had the highest reported code correctness at 62%. Across evaluated LLMs, security exploits succeeded against around half of the programs that were functionally correct, on average. | The exploit result is the benchmark authors’ aggregate finding for these tasks and evaluated models, not a rate for all AI-generated software. |
| FCV-Attack, Google Research / ACL, 2026 | Whether functionally correct agent-generated patches could still contain vulnerabilities. | The study reports a 40.7% attack success result specifically for CWE-538 with GPT-5 Mini + OpenHands. Across 12 agent-model combinations, the reported attack required black-box access and one query. | The percentage belongs to that model, scaffold and vulnerability category; it is not a general attack rate. The study illustrates that functional test success can coexist with a security flaw. |
These benchmarks differ in codebase, task, model, security criterion and evaluation method. For example, exploit execution is a stronger and different signal than a functional test pass. Their value is in exposing specific failure modes, not in producing a common score for “how secure AI code is.”
Rank #3
Do newer models perform better at finding and fixing flaws?
A longitudinal study in Empirical Software Engineering, published by Springer Nature in 2026, evaluated models on real developer prompts from DevGPT at two points: October 2024 and September 2025. It reports improved detection and repair results among the newer models it tested. The figures below describe that study’s prompts and evaluation protocol, not a universal success rate.
| Evaluation point and model | Vulnerability detection | Vulnerability fixing |
|---|---|---|
| October 2024: GPT-4o | Roughly 53–56% | About 50% |
| September 2025: GPT-4.1 | 82% | 75% |
| September 2025: GPT-5 | 78.6% | 78.6% |
| September 2025: Claude Opus 4.1 | 80.4% | 76.8% |
The comparison indicates progress on detection and repair within that dataset; it does not establish that generating secure software is solved. The study’s authors still emphasize manual analysis and static checkers, and warn that AI-generated changes can introduce vulnerabilities or give developers misleading confidence.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Why doesn’t passing the tests prove code is secure?
Functional tests check the behaviors they were written to check. They may confirm that expected inputs produce expected outputs without covering malicious inputs, unintended access, sensitive-data exposure or other security properties. A patch can therefore pass every existing test and still contain a vulnerability.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →The FCV study examined this gap in functionally correct patches produced by code agents. Its attack results are limited to the evaluated examples and agent-model combinations, but they demonstrate why “all tests pass” and “no vulnerability found” are not interchangeable conclusions. The same caution applies to generated applications: a working result is evidence of functionality, not a security audit.
Best Value
How should developers use AI-generated code safely?
Use AI to speed up implementation, review and investigation, but match the checks to the software’s risk. Security should be evaluated explicitly rather than inferred from a successful build or a model’s confidence.
- Define the security requirements before generation. Specify trust boundaries, permitted access, sensitive data and expected behavior for invalid or hostile inputs. Vague functional prompts leave security assumptions unstated.
- Review the change in context. Check how it interacts with authentication, authorization, data storage, error handling and neighboring code. A locally plausible function can still violate a repository’s security assumptions.
- Run security-focused checks as well as functional tests. Use static analysis and tests designed to exercise security properties. Treat a clean result as useful evidence within those tools’ coverage, not proof that no flaw exists.
- Validate consequential findings independently. For higher-risk software, have a qualified reviewer assess the threat model and use exploit-oriented validation where appropriate. Verify an AI-reported issue before changing code, and verify a claimed fix rather than relying on its explanation.
- Keep human review accountable. A reviewer should understand what changed and why the checks are relevant. AI can help surface candidates for investigation, but it should not replace security ownership or review.
The practical lesson is to separate the question “Can this system help find a vulnerability?” from “Can I trust its generated code to be secure?” Evidence for the first does not answer the second; secure use requires checks aimed at security itself.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
