A direct Jev judgment is cheaper and can be safer than a bundle of dimension scores—but it is not always more accurate. Across three classification settings, dimension scores helped on a Japanese task where cues were distributed across the text, and a combined model led on a contextual bookkeeping task. On a synthetic task, a free character n-gram baseline beat the dimension pipeline on leakage-resistant splits. The practical choice depends on the task, the errors that matter, and whether the methods fail on the same examples.
What was compared
In experiments reported by ikkun on 17 September 2026, each row received either one direct Jev classification call or a set of 12–14 narrower Jev scores. The scores were cached and combined using weights fitted locally on labeled examples. The comparisons also included free local baselines, including character n-grams, word-bigram naive Bayes, and a majority-class classifier.
As an Amazon Associate I earn from qualifying purchases.
The three settings were a synthetic B2B-reply classification task, a Japanese natural-language-inference task designed so no single sentence determined the label, and a bookkeeping export. These are distinct tasks and protocols, not a single benchmark with interchangeable scores. In particular, the bookkeeping task included descriptions, amounts, and credit-side account context; it was not classification from bare bank-statement text.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
The author’s dimensions were hand-written for these tasks. Their results do not establish that any arbitrary set of dimensions will help, nor that decomposition beats an already strong direct call.
#1 Best Overall
- Use scikit-learn to track an example ML project end to end
- Explore several models, including support vector machines, decision trees, random forests, and ensemble methods
- Exploit unsupervised learning techniques such as dimensionality reduction, clustering, and anomaly detection
- Dive into neural net architectures, including convolutional nets, recurrent nets, generative adversarial networks, autoencoders, diffusion models, and transformers
- Use TensorFlow and Keras to build and train neural nets for computer vision, natural language processing, generative models, and deep reinforcement learning
How the methods performed
Synthetic B2B replies: random splits overstated the dimension model
The dimension pipeline reached 98.0% on random row folds, but fell to 90.0% when entire template families were held out. Character-bigram naive Bayes scored 93.5% on those grouped folds. The gap matters: if near-duplicate templates land in both training and test data, a model can appear to generalize while relying on familiar wording. The author also described this synthetic task as too easy for the direct question to provide strong evidence about model capability.
Japanese cue-aggregation task: dimensions improved on a weak direct call
The direct Jev call scored 64.7%; 12 scored dimensions with fitted weights reached 74.0% out of fold, a reported difference of 9.0 percentage points (95% confidence interval: 3 to 15 points; p=0.0050). Character-bigram naive Bayes also scored 74.0% without API calls. The labels included noise and were skewed toward neutral, at 55%, so the result is evidence for this task setup rather than a general expected gain from decomposition.
Rank #2
Confidence was not a reliable correctness signal in this case: 126 of 300 rows had direct-call confidence of at least 0.9, and those rows were 72.2% accurate.
Bookkeeping: the stack beat either Jev approach alone
On 2,101 test rows from the author’s accounting data, a stack combining 14 dimensions with 12 n-gram class probabilities scored 0.9695 accuracy. Word-bigram naive Bayes alone scored 0.9491, dimensions with fitted weights scored 0.9105, and a direct 12-choice call scored 0.3998. The setup focused on the top 12 debit accounts and used contextual fields, which limits how broadly those figures should be applied.
The stack’s result illustrates why error overlap matters: combining methods can help when one corrects examples the other gets wrong. A higher aggregate score alone does not tell you whether that advantage will persist on new data or whether the combined system’s mistakes are acceptable.
Accuracy is not the only deployment metric
For a security-text test containing 339 benign rows that mentioned injection techniques, the direct choice method had a 1.5% false-positive rate; the 12-dimension model had a 37.2% rate—about 25 times higher in that test. The author linked the dimension model’s learned behavior in part to surface cues such as obfuscation and hidden content, which also appeared in benign security documentation. These rates describe that test set, not a forecast for another deployment.
Rank #4
If false positives are costly, evaluate them directly on representative hard benign examples. Tune the decision threshold for the application rather than treating a default score cutoff as universally safe. An independent 2026 Jev benchmark paper notes that binary probabilities may rank examples well even when a fixed 0.5 threshold performs poorly on some tasks. That paper is broader context, not a replication of these three experiments.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Choose a method with a measured workflow
- Define the target and error costs. Decide whether accuracy, recall, precision, false-positive rate, or a particular operational cost matters most. Keep hard benign examples in the evaluation set if false alarms are consequential.
- Build a free baseline. Try an n-gram or TF-IDF classifier and a simple majority baseline before paying to extract model features. Use splits that reflect deployment: group related templates, customers, or sources where leakage is possible.
- Test one direct question. Measure it on held-out examples. Keep the direct call if it meets the target and its errors are acceptable; extra dimensions add calls, tokens, and opportunities to learn misleading cues.
- Try dimensions only when there is a reason. They are worth testing when the direct call is weak and relevant evidence is distributed across cues. Fit weights on training data only, and evaluate out of fold or on a separate test set.
- Compare row-level errors before stacking. Check whether the baseline, direct call, and dimension model correct different examples. Combine them only if the improvement survives held-out evaluation and the resulting false-positive burden is tolerable.
- Recheck after model or prompt changes. The independent benchmark evaluated pinned Jev version 1.13.0, while the experiments discussed here used the moving
jev-latestalias. Results from the two studies use different protocols and should not be treated as direct confirmation of one another.
What the reported costs do—and do not—say
For the author’s experiment, ikkun reported 34.1 million input tokens across 5,477 test rows and 25,174 Jev calls, with $1.43 in input cost using the then-stated rate of $0.042 per million input tokens. Output tokens were counted separately and not priced. The article estimated $19–26 per million rows for one direct question and $42 for twelve dimensions. These are reported prices and workload estimates, not independently verified current pricing; actual cost depends on prompts, token volume, output, and applicable rates.
Best Value
The independent benchmark by Tobias Deußer, Lorenz Sparrenberg, and Rafet Sifa evaluated Jev version 1.13.0 across 37 datasets and 346,009 requests. It used one prompt template per dataset, ran each request once, and did not measure run-to-run variance; its authors also note that the jev-latest alias can move to newer versions. Those limitations make the benchmark useful as broader context, but not a guarantee that a particular prompt, version, or deployment will reproduce its results.
When should you split a judgment into scored dimensions?
Use dimensions as an experiment, not a default architecture. The evidence here supports them most clearly when a direct call is weak, the task’s cues are distributed, and leakage-resistant evaluation shows improvement. A cheap local model may match or beat them, and a direct call may have a much safer error profile on sensitive cases. The decisive check is not simply which method has the best headline accuracy, but whether its mistakes differ in useful ways and remain acceptable for the deployment.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
