Does your AI code reviewer read CLAUDE.md? A review comment that names a rule and a line range looks like proof that it did. Daniel Nwaneri’s test, published September 9, 2026 and corrected September 10, 2026, shows why that look is misleading. A citation shows that a rule text was surfaced during review. It does not show that the rule drove the finding, and in his follow-up, the cited line ranges still pointed to the rule’s old position after he moved it. The question that matters is whether the reviewer can show which current line it read.
The setup
The test repository, rules-demo-api, is a small Cloudflare Worker that accepts feedback and logs a rating. It contains matching instructions in CLAUDE.md and AGENTS.md, so each rule appears in both files. Two rules carry the test:
![Claude AI for Beginners Bible: [5 in 1] The Ultimate Guide to Automate Your Work, Save Hours Every Week, and Use AI for Real-World Results](https://m.media-amazon.com/images/I/415+fSJacsL._SL160_.jpg)
As an Amazon Associate I earn from qualifying purchases.
- Logging rule:
Never log request headers or bodies. They may contain secrets, tokens, or PII. - Route rule:
All new route paths must be kebab-case (/feedback-summary), never camelCase (/feedbackSummary). This is an internal convention, not a general best practice.
The duplication is deliberate and important. When the same sentence sits in two files, a citation to it cannot tell you which file the reviewer used, or whether one file overrode the other.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesCase one: a risk the reviewer could spot anyway
The author added code that logged request headers and body. Both Qodo and CodeRabbit flagged it. The author himself treats this as a weak test of file-reading, and the reason is simple: logging request bodies is recognizably risky without any repository instruction. A catch here shows that the tool noticed a dangerous pattern. It says nothing about whether CLAUDE.md or AGENTS.md was opened.
#1 Best Overall
Case two: a route rule that matches common practice
The second change added a camelCase route, /feedbackSummary, which violates the kebab-case rule. The three reported outcomes differ in a way that matters:
| Reviewer and setting | Reported result | Cites an instruction file or rule? |
|---|---|---|
| Qodo | Flagged the route and reasoned about a mismatch with the route convention | Yes, cited the kebab-case rule |
| CodeRabbit, Chill profile | No actionable comments | Not applicable; no finding was produced |
| CodeRabbit, Assertive profile | Flagged the naming issue | No reference to either instruction file |
The Assertive result is the hardest to interpret. Kebab-case is a widespread convention, so a reviewer could reach the same finding from general knowledge. The author concedes this point, which means the finding does not establish that CodeRabbit read the local rule. Nwaneri’s summary of the two tools is pointed: “Both tools caught the second one eventually. Only one of them could tell me why.”
Case three: moving the rule
A reader, pm25coder, pointed out that a citation proves a rule was surfaced, not that the file was read when the decision was made. Because the rule was duplicated in both files, the test also could not say which file took precedence. The reader suggested moving the rule and rerunning the same review.
The author moved the rule seven lines down in both files and reran Qodo. The finding still cited the old line ranges. According to the discussion, the rerun had a current commit anchor, but the line labels in the prose were stale, so the reference was partly current and partly wrong. CodeRabbit had not been rerun against the later commit, so the test has no comparable post-move result for it.
What a citation does and does not establish
Read a reviewer’s citation as a claim that needs three checks before you trust it:
- Surfaced: the rule text was in the reviewer’s context. This is all a citation reliably shows.
- Causal: the rule produced the finding. A citation does not establish this, and the duplicated rule in this test rules it out as evidence for either file.
- Current: the cited span still holds the rule it names at the commit under review. The moved-rule rerun shows that a citation can be stale even when a commit reference looks current.
As Nwaneri puts it: “A citation proves a rule was surfaced, not that it drove the decision or that it’s still accurate.” The reader’s compressed version is “A citation is not evidence that the file was read at decision time.”
How to run a stronger check
The discussion proposed the following test. None of these steps was completed in the reported experiment; they are the checks the discussion recommended.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →- Put a unique sentinel phrase, one that appears nowhere else in the repository, in
CLAUDE.mdonly. - Put a conflicting rule in
AGENTS.mdonly, for example a different route style, so the two files disagree on a single point. - Open a change that exercises both rules and record every citation the reviewer returns, including the file name and line span.
- Resolve each citation against the exact commit reviewed, for example with
git show <commit-sha>:CLAUDE.md | cat -n, and confirm the line numbers match. - Assert that the cited span contains the rule the citation names. A citation that names the sentinel but points to a span without it fails.
- Move the rule, commit, and rerun. Confirm the new citation points to the new span, and record whether the reviewer reran on the new commit at all.
- Where possible, use checkable requirements instead of style prose. A required literal string, or a route pattern such as
^/[a-z0-9]+(-[a-z0-9]+)*$, can be verified mechanically. A prose instruction such as “prefer kebab-case” cannot.
What this test does not show
- It is a self-reported, single-repository example with two tools. It is not a general performance comparison.
- Profile settings changed what was reported. The Chill profile produced no actionable route comment, and the Assertive profile produced one.
- The account does not verify current Qodo or CodeRabbit documentation, profile defaults, or behavior. Check those directly before relying on any specific setting.
The practical takeaway for teams is narrow. Before you accept a reviewer’s rule citation as evidence, resolve it against the reviewed commit and check that the text is there, and treat a finding that matches common practice as unproven about your local rules.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
