In CauterRule v0.3.0, the semantic matcher reportedly required a cosine similarity of at least 0.80, while a clean paraphrase signature scored 0.631. That gap meant the semantic channel could remain silent on the very paraphrases it was intended to catch. Debashish Ghosal’s account of the v0.3.1 changes shows why a similarity threshold should be chosen from observed match scores—not from intuition—and why the reported fix is not proof that 0.62 will work everywhere.
How a high floor silenced the semantic channel
Ghosal describes CauterRule v0.3.0 as combining token F1, bigram similarity, and MiniLM cosine similarity with weights of 0.5, 0.3, and 0.2, respectively. The semantic component was gated by a cosine floor of 0.80: scores below that threshold did not contribute.
As an Amazon Associate I earn from qualifying purchases.
The article reports a clean paraphrase-signature cosine of 0.631—below the 0.80 floor. When the failure_class label, such as ci/lint, was included in the embedded text, the reported score was lower still: 0.547. In the author’s account, this meant the semantic code existed but did not activate for the paraphrase cases it was supposed to help match. These are figures reported by Ghosal, not results independently reproduced here.
Recommended Free Tools
The blend’s weights explain why the gate mattered. In the article’s short-paraphrase example, there were no shared tokens, so token F1 contributed nothing; if the semantic component also failed its gate, the remaining bigram component could not be supplemented by semantic similarity. Even when semantic similarity is allowed through, its nominal weight is 0.2. A channel that rarely activates—or contributes only a limited share of the blend—may not move a result across a promotion threshold.
What changed in v0.3.1
Ghosal says v0.3.1 made three matching changes. The article reports no change to the model or prompt and no change to the blend weights.
- Lowered the semantic floor: from 0.80 to 0.62.
- Compared two signature views: one without the class label and one with it, then used the higher similarity.
- Changed the embedded representation: used a structured failure signature instead of whole-trajectory prose.
Together, these changes address two distinct issues in the account: a threshold above the observed positive score and text content that could dilute the similarity signal. The article presents them as a combined change, so it does not isolate how much each one contributed to the reported outcomes.
What the reported evaluation shows—and does not show
Ghosal reports the following golden-recall values, with the paired groups labeled “gpt / llama” in the article. The figures come from the article, not an independently inspected underlying report.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
| Version | Golden recall (gpt / llama) |
|---|---|
| v0.3.0 | 0.170 / 0.228 |
| v0.3.1 | 0.377 / 0.427 |
The author characterizes the first group’s increase, from 0.170 to 0.377, as nearly doubling. The figures suggest a substantial reported recall improvement, but they do not establish that lowering the floor alone caused it: the signature views and representation changed at the same time.
The article also reports v0.3.1 golden pass rates of 82% [0.70, 0.89] and 83% [0.72, 0.91], each with n=60 and Wilson confidence intervals. Ghosal says these clear a ≥70% gate. For v0.3.0, the article gives a golden pass rate of 30–50% with n=10 and no confidence interval. Because the sample sizes and reporting differ, these figures should not be read as a controlled, like-for-like estimate of the size of the improvement.
Reference expansion is reported as 19/303 before the changes and 201/303 and 198/303 afterward, described by the author as about tenfold. Ghosal cautions that outcomes for adapters and raw/ci reflect multiple simultaneous changes and that the article does not separate attribution that was not instrumented. The source is Debashish Ghosal’s September 15, 2026 article, “A Floor of 0.80 and a Ceiling of 0.63: The Semantic Channel That Never Fired”; the underlying field reports and implementation changes were not independently examined for this account.
How to calibrate a similarity floor
A threshold is meaningful only in relation to the score distribution for the matching task at hand. A floor set above many real positive examples silently suppresses true matches; a floor set too low can admit unrelated examples. Calibration should therefore consider positive and negative cases together, not just whether a few known paraphrases clear a chosen number.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →- Collect representative labeled examples. Include known-correct matches and plausible non-matches from the corpus where the matcher will run. Keep evaluation examples separate from examples used to tune the threshold.
- Log raw scores before gating. Record the cosine values and the other component scores, along with the representation used to produce each embedding. Without pre-gate scores, a channel that never activates can look like a channel that simply has little influence.
- Inspect positive and negative score distributions. Look for overlap, not just a single maximum or minimum. The useful floor depends on how often true matches fall below it and how often non-matches rise above it.
- Evaluate the full decision, not cosine alone. Measure false positives and false negatives under the actual blend and promotion rules. A semantic term weighted at 0.2 can behave differently depending on the other components and downstream cutoff.
- Validate on held-out examples. Choose a threshold on one set, then assess it on examples not used to tune it. If the same examples determine and validate the threshold, reported performance can overstate how well it generalizes.
- Recalibrate when the setup changes. Corpus, embedding model, signature text, and preprocessing can all affect score distributions. A floor measured for one configuration is not automatically suitable for another.
Is 0.62 portable?
The article does not establish that 0.62 transfers across corpora, models, or embedding configurations. It reports that the floor was calibrated on one corpus/model setup and leaves open whether thresholds should be set per corpus. In practice, treat 0.62 as a reported setting for this change—not a general MiniLM cosine standard—and check it against known positives and negatives in the deployment corpus.
Should the floor be per-corpus, and should the semantic weight change?
Those are separate calibration questions. A corpus-specific floor may be warranted if its positive and negative score distributions differ from those of another corpus, but the article does not settle that policy. Nor does it establish whether increasing the semantic weight above 0.2 would improve outcomes. Changing a floor affects which scores enter the blend; changing a weight affects how much an admitted score influences the final result. Evaluate them separately where possible, using held-out examples and the matcher’s actual verdict criteria.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
