In Christian Anderson’s 62-case test of claims in product descriptions and posts, combining Jev with DeepSeek caught errors each checker missed alone. His combined pass rule got 61 of 62 cases right, according to his report. That is a promising result for his publishing workflow—not proof that two checkers will improve accuracy everywhere.
What Anderson tested
Anderson wanted to check whether a product description or post said more than its source material supported. He assembled 62 cases from actual Gumroad product files and his DEV posts: 22 claims were supported by their sources, while 40 went beyond what the sources established.
As an Amazon Associate I earn from qualifying purchases.
He ran the cases separately through two systems. DeepSeek (deepseek-v4-flash) read the source and returned PASS or FAIL. Jev (typesafe/jev-1.13) returned a probability that a claim was supported. Anderson then compared different pass rules against his labels for the cases.
This was a small, specific test reported by its author. The article does not provide public raw cases and code for independent reproduction, nor an independent audit of the labels. The findings should be read as a report of what happened on this set, not a validated benchmark.
#1 Best Overall
How the checkers performed on the 62 cases
In the table, a true positive (TP) is a supported claim passed; a false positive (FP) is an unsupported claim passed. A true negative (TN) is an unsupported claim rejected; a false negative (FN) is a supported claim rejected. “No answer” records DeepSeek’s non-responses. Accuracy is the author’s reported accuracy among answered cases.
| Checker or rule | TP | FP | TN | FN | No answer | Accuracy (answered) | Time |
|---|---|---|---|---|---|---|---|
| DeepSeek chat | 22 | 2 | 35 | 0 | 3 | 96.6% | 21.5 s mean |
| Jev, pass at p ≥ 0.5 | 22 | 4 | 36 | 0 | 0 | 93.5% | 0.35 s mean |
| Jev, pass at p ≥ 0.9 | 21 | 0 | 40 | 1 | 0 | 98.4% | 0.35 s mean |
| Both combined | 22 | 1 | 39 | 0 | 0 | 98.4% | Not stated |
These are Anderson’s reported figures for this run. He also reports a 0.31-second median for Jev versus 20.6 seconds for DeepSeek, three DeepSeek non-answers, and a total cost of $0.0018 for all 62 Jev checks. Those timings and cost describe the reported run; they are not current pricing or guaranteed latency.
Why a second checker helped in this test
The useful finding is not simply that one system scored higher. The systems made different mistakes, so a policy that rejected a claim whenever either checker rejected it reduced unsupported claims getting through.
For example, DeepSeek passed a claim that a holiday pricing guide would help users “save at least £25,” while Jev gave it a support probability of 0.13. In the opposite direction, at Jev’s 0.5 cutoff it passed four unsupported claims. Three were product descriptions that overstated coverage, and DeepSeek rejected those three.
Rank #3
Those examples explain the benefit of combining checks on Anderson’s cases: one checker sometimes caught what the other let through. They do not show that the errors will be complementary in a different dataset, subject area, or publishing workflow.
The pass policy Anderson adopted
Anderson’s operational rule was to fail a claim if either checker said FAIL. If DeepSeek returned no answer, Jev had to score the claim at least 0.8 for it to pass. In his account, that combined policy got 61 of 62 cases right.
The remaining error was an unsupported description of one of his posts: DeepSeek passed it, and Jev scored it 0.63. This illustrates an important policy choice. Raising the Jev threshold can reduce unsupported claims passed, but on this set Jev at p ≥ 0.9 also rejected one supported claim. A threshold is a trade-off, not a universal setting.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
What to weigh before copying the approach
For a workflow like Anderson’s, a second checker is worth considering when the cost of publishing an unsupported claim justifies additional checks. Decide the pass policy against the errors that matter most to your work, rather than adopting a threshold based on one small sample.
Best Value
- False positives: How often does the system pass a claim the source does not support?
- False negatives: How often does it reject a claim that the source does support?
- Non-answers: Does a missing response block publication, trigger another check, or invoke a separately defined fallback?
- Threshold: What probability is high enough to pass, and what errors does that choice trade?
- Repeatability: Anderson says two Jev runs on all 62 cases differed by at most 0.04, with an average movement of 0.007. That is evidence about those repeated runs, not all inputs or future versions.
- Cost and latency: Measure these in your own environment; Anderson’s reported figures describe his particular run.
- Representativeness: Test on claims and source documents resembling your own, with labels you can justify. A result on product files and posts may not carry over to other domains.
Claim checking is not the same as model routing
The tested task was deciding whether a claim was supported by a source—not choosing which model should handle a request. Jev’s routing documentation describes typed outputs such as a finite model choice, a complexity score, and a probability that tools are needed. It also describes jev-router, an open-source, OpenAI-compatible LiteLLM proxy: it summarizes incoming messages, filters candidate models by capability, then lets Jev choose. The documentation says it has a rules-based cheapest-eligible fallback when no key is set. Those product capabilities are distinct from Anderson’s claim-support experiment and do not validate its results for routing.
A separate paired and self-audited evaluation by Jiawei Li, dated October 1, 2026, examined Jev and Laya across 11 agent decision points. Its abstract reports that Jev was significantly more accurate on nine points, but neither system beat chance on zero-shot model routing, and both tied on RAG relevance gating. The paper also says errors in an earlier analysis distorted deployment claims. That is a different evaluation, but it reinforces why results from one task should not be generalized to another: Li’s October 2026 evaluation.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
