When several WhatsApp alerts are waiting, attaching a customer’s reply to the newest one can be wrong: “when will the parcel arrive?” may refer to an earlier delivery alert, not a newer viewing notification. Danish Javed describes inserting a fixed-choice classifier into ChatRail’s matching flow so it could choose the relevant alert—or abstain—before the existing newest-alert fallback. In his small, self-designed tests, all three compared models selected the right alert in 93% of 14 cases; Jev’s reported advantage was speed and cost, not higher accuracy.
Why the newest-alert rule could misread a reply
ChatRail sends WhatsApp alerts about events such as dispatched orders, booked viewings and invoices due. A customer may respond to an alert later, after other alerts have arrived. If several alerts are waiting, simply linking an unthreaded reply to the newest one can attach it to the wrong conversation.
As an Amazon Associate I earn from qualifying purchases.
For example, a customer asking “when will the parcel arrive?” could mean an earlier dispatch alert, even if a newer alert concerns a viewing. The issue is not that the reply lacks meaning; it is that recency alone does not establish which alert it refers to.
Where the classifier fits in the matching flow
Javed’s implementation keeps the straightforward cases deterministic and uses Jev only when multiple alerts are open. Its order of operations is:
- Use the WhatsApp reply button when available. An explicit reply reference takes precedence.
- Use the sole waiting alert when only one exists. There is no need for a model to disambiguate it.
- Ask Jev when several alerts are waiting. The question is: “Which of these alerts is this message about?” The model chooses one alert or “none of these.”
- Accept a selection only at the configured confidence threshold. In this implementation, a score of at least 0.8 records the choice as
model_choicewith its probability. - Fall back to the existing newest-alert guess if needed. The fallback applies when confidence is below 0.8, the response takes longer than two seconds, or an error occurs. If no alert is waiting, the system links nothing.
This arrangement lets a model handle the ambiguous case without making incoming messages depend on the model being available. Javed said the feature was live behind a flag, and that only connections already using managed AI sent messages to Jev.
What the reported comparison found
Javed compared Jev with Gemini 2.5 Flash Lite and GPT-5.6 Luna. The figures below are the author’s results, not independently verified measurements or general performance guarantees.
| Measure | Reported result | How to read it |
|---|---|---|
| Alert-selection accuracy | 93% for Jev, Gemini 2.5 Flash Lite and GPT-5.6 Luna, each in a 14-case test — Danish Javed, 2026 | The author wrote the cases himself; many were designed to expose the old recency rule. |
| Cases where newest alert was wrong | All three models got all five cases right; the old rule got none right — Danish Javed, 2026 | This is a five-case subset of the same small test, not a separate benchmark. |
| Median latency | Jev: about 350 ms; Gemini: about 640–1,000 ms; GPT: about 2.1–2.6 seconds — Danish Javed, 2026 | Measured end to end from the author’s machine through OpenRouter, over two runs; network time was included. |
| Estimated cost per 10,000 messages | Jev: about $0.25; Gemini: about $0.41; GPT: about $1.20–$1.30 — Danish Javed, 2026 | The author reported charges from OpenRouter. These are estimates for the stated workload, not a universal price per issue. |
In a simulation, the old rule linked 3 of 12 contacts correctly versus 12 of 12 Jev links in a simulation reusing earlier test alerts — Danish Javed, 2026. Because the simulation reused alert texts from the first test, it does not provide an independent sample. Javed also reported that the 12 Jev calls cost $0.00027 across 12 Jev calls — Danish Javed, 2026. That total describes those calls in that simulation; it should not be treated as a fixed cost for every future matching task.
Why these results are not production proof
- The sample was small and author-written. The 14 cases were created by Javed, with many chosen to challenge the old rule.
- The simulation reused test material. Its 12-contact result reused the earlier alert texts rather than testing new, independent examples.
- The problem had not yet appeared in production. Javed wrote, “We currently have zero production messages that hit rule 3.” The result therefore demonstrates behavior in constructed cases, not a measured reduction in live mismatches.
- The confidence cutoff was not validated. Javed chose 0.8, but every Jev selection in the reported runs scored at least 0.97, so those runs did not test what happens near the cutoff.
- Equal accuracy does not establish broad equivalence. Each model matched at 93% on this particular test. The results do not prove that they perform equally on other customers, message patterns or alert sets.
Alert matching is different from answering the customer
Choosing which alert a reply refers to is a distinct task from deciding whether the alert contains enough information to answer the customer. In Javed’s separate answerability test, Jev scored 71%, while GPT scored 93–100% across two runs. He left that task to the reply model. Those figures concern answerability, not alert selection, and should not be combined with the 14-case matching comparison.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What to evaluate before using this pattern
The useful design lesson is to treat model-based matching as one branch in a guarded decision flow, not as a replacement for explicit references or simple deterministic cases. To assess whether it suits another system, measure the parts that matter to that system:
- Accuracy on representative, independently assembled cases, including replies that refer to an older alert and cases where no alert fits.
- End-to-end latency under the actual network and service path, not model response time alone.
- Cost for a clearly defined message volume and workload.
- Whether the model can abstain when none of the candidate alerts is relevant.
- What happens on low confidence, timeouts and errors—and whether the fallback is safer than leaving the reply unmatched.
Javed’s reported comparison supports a narrow conclusion: in his test, the classifier corrected the five examples where recency failed, while the three models had the same overall alert-selection accuracy. Its reported case for Jev was lower latency and cost. The small, constructed test and absence of production cases leave real-world accuracy unresolved.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
