What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Zero output tokens means the model does not decode a text answer. It still processes the request in a forward pass, then reads hidden states at answer positions to choose among options declared by the caller. The phrase describes how the result is produced—not an absence of computation.
How can a model answer without generating tokens?
In the approach described by Zehua Cheng, Wei Dai, and Jiahao Sun, an application supplies a state and one or more questions. Each question includes a defined set of allowed answers, such as named choices, an ordered score, or true and false. The request assigns a position to each answer, and the model evaluates the rendered request in a single forward pass.
As an Amazon Associate I earn from qualifying purchases.
Instead of generating a word or sentence, the model reads hidden states at those designated positions and applies a softmax over the permitted answers. The result is a probability distribution over the declared options. Because the output interface only represents those options, it cannot return an answer outside the set through that head. The authors say multiple questions about one state can be handled in the same pass.
The paper’s title uses “multimodal,” but its request can be a string or a compactly serialized JSON value, and its examples and reported benchmarks focus on structured decisions and map-like environments. It does not establish performance across image, audio, or video tasks generally.
#1 Best Overall
What does zero output tokens change?
A conventional generative model decodes text, after which application code may need to interpret or parse the response. The authors argue that this path can add latency, token cost, and failure cases such as malformed or missing answers. A typed decision interface avoids decoding that open-ended response and gives the application a result in the form it expects.
That can suit bounded software decisions where the application already knows the possible answers and can send uncertain cases elsewhere. It does not make the model’s reasoning free or instantaneous: the input still has to be processed, and the model still performs computation to produce its distribution.
Rank #2
What performance did the paper report?
Cheng, Dai, and Sun report a latency of 30.9 ms per decision and throughput of 32 decisions per second on one consumer GPU in their setup. These are measurements from the paper, not guarantees for other hardware, software stacks, request shapes, or production conditions. The authors also compare their approach with hosted-model measurements, but the configurations and cost bases differ. The paper, published on arXiv on September 20, 2026, is the primary source for its method and reported results: this-that-model-1.0.
Free tools Windows power users keep installed
One-click scans. No signup required.
How strong is the accuracy evidence?
On a third-party recorded cohort of 68 decision questions, the authors report accuracy of 0.941 and a Brier score of 0.042 for this-that-model-1.0. Jev scored 0.765 accuracy and 0.133 Brier score on the same items. The authors qualify the comparison: the cohort is small, its wording came from the third party, and the accuracy gap rests on 12 questions. It is not broad evidence that the model is more accurate than hosted frontier models in general.
The paper also evaluates 7,305 questions across 15 families and two environments. Scores vary by task, and the authors identify map-wide search as a persistent weakness. On a stochastic-actuator family, the reported probability-quality score was 0.750 against an estimated ceiling of 0.746. That result describes this constructed evaluation; it is not a universal calibration guarantee.
Where does this approach fall short?
A constrained answer interface does not substitute for a method that can carry out complex reasoning or search. The authors report a score of 0.560 on multi-step arithmetic, compared with 0.98 to 1.00 for the hosted systems they cite. They attribute the gap to the model’s single forward pass, which cannot carry intermediate results through the calculation. For questions that require searching across a map or otherwise exploring possibilities, the paper’s conclusion is to use a method that performs the search.
- Good fit: a bounded classification or decision with known answer options, especially when a typed result and probabilities are useful.
- Poor fit: a task that needs multi-step arithmetic, intermediate working, or broad search.
- Deployment choice: the paper describes an open-source software model and inference code, not a consumer hardware product. Local execution and a hosted service differ in deployment and data handling; the paper’s performance claims alone do not independently verify operational behavior in any particular deployment.
What “zero output tokens” does—and does not—tell you
It tells you that no textual answer is decoded: the system returns a typed choice based on hidden states and a caller-defined option set. It does not tell you that the model did no work, that it can answer arbitrary questions, or that it will outperform a generative model on every task. As the authors put it, “A decision is not a document.”
Recommended Free Tools
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
