In one test reported by developer Tony Dzi, three of four language models produced specific code-review findings after receiving repository paths but not the files’ contents. The fourth said it lacked the data. It is a cautionary account, not a benchmark: the post names none of the models and does not publish their raw responses. Its practical lesson is clear: a model cannot inspect source code it was never given.
What happened when the models received paths instead of code?
Dzi says that on August 10, 2026, he asked four models to review code by giving them repository paths rather than the source itself. Three returned findings; one reported that it had no data. In the post, he describes the three responses as fabrications, including references to nonexistent functions, treating a file as though it were written in another programming language, and suggesting command-line flags that did not exist.
As an Amazon Associate I earn from qualifying purchases.
That is the author’s account of a single run. The post does not identify the models or include the outputs, so readers cannot independently check the examples or use the result to compare vendors. The reported count is three of four—75% of that run only, not an estimate of how often AI reviewers generally invent findings. Dzi’s September 21, 2026 post on DEV Community describes the experiment.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesWhy a file path is not review input
A path tells a model where a file might be on a particular machine; it does not transmit that file’s bytes. Unless the review system has a tool or integration that can access the repository and retrieve the file, a prompt such as “review src/server.js” supplies a location, not the code. A model’s confident, well-formatted answer is not evidence that it opened the file.
Dzi’s blunt formulation is: “A reviewer that cannot see the code does not say ‘I cannot see the code.’” His experience does not establish that every model will behave this way. It does show why the workflow should verify what the model actually received instead of relying on the model to recognize and disclose missing input.
How to make an AI code review auditable
Send the artifact, not just its address
Provide the relevant file contents directly, or use an integration that retrieves them and confirms delivery. For a large file or review context, divide the material into whole, clearly identified parts rather than silently omitting sections. The important check is that the reviewer received the intended source—not merely a filename or a prompt describing it.
Check the transport when inputs are large
Dzi reports hitting “Argument list too long” when passing about 82 KB of context as a shell argument, which he attributes to bash. That figure describes his experience, not a universal size limit. If a wrapper or command-line pipeline runs into an argument-size error, pass the context through files that the wrapper reads rather than placing the entire payload in an argument.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Verify each finding before changing code
Treat a model’s report as a claim to investigate, not an instruction to execute. Dzi says his practice is to reproduce each finding or reject it with a written reason. That record makes it possible to distinguish a real defect from an incorrect assumption about the code or system.
Test whether the proposed fix matches the system’s contract
One example in the post concerns a health check: a vendor reportedly suggested counting only a 2xx response as proof that a daemon was alive, while the service’s root path deliberately returned 404. Under that rule, a running service could appear dead and trigger a disruptive restart. The relevant question was not whether 404 is generally healthy, but whether that endpoint’s documented behavior matched the proposed check.
Use a panel only when the task justifies it
Dzi says his panel uses models from multiple vendors because he believes models from the same family may fail in correlated ways. That is his rationale, not a controlled comparison showing that a multi-vendor panel is more accurate. He also says trivial typo fixes do not need a four-vendor review and that a single-vendor run should be disclosed as such. The appropriate setup depends on the risk and complexity of the change; more reviewers do not replace access to the code or verification of findings.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What the experiment does—and does not—show
The post offers a concrete warning about one failure mode: a reviewer may produce plausible findings without having received the artifact it is supposed to inspect. It does not establish a general hallucination rate, show which vendor is most reliable, or prove that all models respond this way to paths. Dzi reports that the workflow runs across agent sessions on five machines, but that is a description of his own operation, not independently verified adoption.
Two of the author’s principles capture the operational stakes: “Panel findings are inputs, not orders,” and “An instrument that over-reports is worse than no instrument, because it justifies action.” In practice, that means checking input delivery, reproducing suspected defects, and recording why a finding was accepted or rejected before making a code change.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
