DriversRecommendedOutdated drivers can make a good PC feel brokenScan driver issues before chasing fixes manually.Scan NowOctober DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsSlow PC?RecommendedPC slow today? Run a repair scan before it gets worseResolve common Windows issues and optimize system performance.Scan Now×
Skip to content
World desk3 min

I Mined 45 Ruff Review Comments for Unwritten Rules. One Held Up; One Failed.

A scan of 45 human review comments across 15 Ruff pull requests produced two candidate rules—but neither established a team convention. Here’s what the author says the failure revealed about clustering feedback.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

A scan of Ruff pull-request feedback produced two plausible candidate rules—but neither was ready to enforce. In the author’s account, the apparent “async” rule drew both of its supporting comments from one pull request, while a second cluster about error messages was too vague to act on. The useful result was not a discovered team policy; it was a lesson in how easily a small sample can make review feedback look more consistent than it is.

What the Ruff review-comment scan found

Ofer’s Instinct Bot says PR Rulebook scanned 15 merged pull requests in astral-sh/ruff. After excluding bot comments, the run contained 45 inline human review comments. Its first clustering pass produced two candidate rules. These are figures reported by the tool’s author for this run, not independently verified measurements or statistics about Ruff’s review practices.

As an Amazon Associate I earn from qualifying purchases.

The candidates differed in how actionable they looked, but neither established a convention across separate pull requests:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Candidate Evidence described by the author Why it did or did not hold up
Include async in a diagnostic annotation when it explains why the diagnostic fires Two accepted-change signals; an 82% cluster score. Both comments came from the same PR. Coherent wording, but only one review conversation. It did not demonstrate recurrence across PRs.
Quote or improve an error message Two comments; one accepted-change signal; a 68% cluster score. The wording was too vague to translate into a specific, enforceable rule.

The reported percentages are cluster scores from PR Rulebook, not probabilities that a rule is true. In particular, a high score cannot make two comments from one PR equivalent to evidence from two independent PRs.

Why the apparent rule failed its recurrence test

The “async” suggestion looked like the stronger candidate because its comments pointed toward a recognizable instruction. But both came from one pull request. They could reflect the same reviewer, context, or discussion rather than a recurring expectation across the project. The author therefore treated them as one review conversation—not proof of an unwritten team rule.

The error-message cluster had a different problem: even with comments about related wording, “quote or improve the error message” did not specify what change a contributor should make or when. A candidate rule needs more than similar language; it must be clear enough that a person can apply it consistently.

What PR Rulebook changed after the run

Ofer’s Instinct Bot says the revised process requires evidence from at least two distinct pull requests before a pattern qualifies as recurring. The author also reports removing fenced GitHub suggestions before clustering, while preserving identifiers inside inline code, and canonicalizing a small set of review concepts before calculating cosine similarity over normalized terms. A regression test is said to reject repeated comments confined to one PR. These are descriptions of the implementation from its author, not independently inspected or validated results.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The changes address two ways the first pass could be misleading: multiple comments in one discussion can resemble broad recurrence, and unrelated comments can appear similar because they share placeholder text inside fenced suggestion blocks.

Rank #3
Sale
The Big Book of Tricks for the Best Dog Ever: A Step-by-Step Guide to 118 Amazing Tricks and Stunts
  • Book: the big book of tricks for the best dog ever: a step-by-step guide to 118 amazing tricks and stunts
  • Language: english
  • Binding: paperback

Candidate rules still need human approval

PR Rulebook is described as a local TypeScript CLI that scans merged pull requests for recurring human review comments followed by code changes. It emits ranked candidate rules with evidence links and confidence scores, with output formats for Cursor .mdc, Claude Code markdown, CodeRabbit YAML, and JSON. The intended role is to help a reviewer find patterns worth examining, not to turn review comments directly into policy.

As Ofer’s Instinct Bot puts it: “Confidence scores rank what a human should inspect. They do not make weak evidence true.” Each candidate still needs a person to check whether the evidence spans separate PRs, whether the requested change was accepted, and whether the rule is precise enough to guide future work.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Availability and limits of the account

The post, dated September 20, 2026, describes PR Rulebook as not yet published on npm and invites five public repositories with active human PR review to participate in a pilot. Its from-source instructions call for Node 20 or later, a read-only GitHub token, and building the CLI to produce a Markdown output file. The author says repository contents and comments are sent from GitHub to the user’s machine and that PR Rulebook has no server. These availability and architecture details are the author’s description, not independent verification.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The post does not provide a controlled benchmark, a comparison with a baseline, precision or recall measurements, or evidence that the two-PR threshold generalizes to other repositories. It describes one tool run and the builder’s response to its failure modes. It should not be read as proof of Ruff’s team-wide conventions or of code-review practices generally.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
PC Slower Than It Used to Be?Free scan - under a minute
Crashes, No Sound, or Screen Glitches?Free driver scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.