Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesIt has been reported, but it is not a plug-and-play RTX 4090 result. A 2026 community benchmark describes Qwen3.8-Flash-Next generating tokens at a median 110.8 tokens per second in a text profile on one 24 GB RTX 4090. That system also used a Ryzen 7900, 192 GB of RAM and an unpruned GSQ IQ3_S quantization. The result is a configuration-specific measurement, not a speed guarantee for every 4090 or a standard NVIDIA deployment recipe.
What the 100 tokens-per-second claim measures
Here, “100T/s” means roughly 100 generated tokens per second, conventionally written as tokens/s or tok/s. It refers to decode: the rate at which the model generates output after processing a prompt. It does not mean 100 trillion operations per second, nor does it describe how quickly the prompt is processed.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
VIPERA NVIDIA GeForce RTX 4090 Founders Edition Graphic Card | $4,649.99 | Buy on Amazon |
| 2 |
|
ASUS TUF Gaming GeForce RTX™ 5080 16GB GDDR7 OC Edition Graphics Card | PCIe 5.0, HDMI/DP 2.1,... | $1,899.99 | Buy on Amazon |
As an Amazon Associate I earn from qualifying purchases.
The community project reports about 4,700–4,800 tokens per second for uncached 32K-token prompt processing, or prefill. That is a different phase and metric from decode. Neither number alone tells you the time to a useful first response or how quickly a complete answer will finish.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
What the RTX 4090 benchmark actually reports
AnnoyingTechnology’s 2026 community repository describes a system with one 24 GiB RTX 4090, a Ryzen 7900 and 192 GB of system RAM. Its baseline used an unpruned GSQ IQ3_S configuration. The repository says the decode cells used six diversified, 512-token, low-reasoning runs.
#1 Best Overall
- 16,384 NVIDIA CUDA Cores
- Supports 4K 120Hz HDR, 8K 60Hz HDR and variable refresh rate as indicated in HDMI 2.1A
- New streaming multiprocessors: up to 2x power and power efficiency
- Fourth generation tensor cores: up to 2x AI power
- Third-generation RT cores: up to 2x ray tracing performance
| Reported result | Profile or condition | How to interpret it |
|---|---|---|
| 110.8 tok/s median decode | Text profile; six diversified 512-token low-reasoning runs in the reported baseline | A result from this project’s specific hardware and configuration, not a general RTX 4090 benchmark. |
| 97.65 tok/s median decode | GPU-vision profile in the same baseline | Below 100 tok/s in that baseline; do not treat it as the text-profile result. |
| 120.10 tok/s decode at 32K; 112.55 tok/s at 128K | Later matched context probes reported separately by the repository | Context-specific results from the project, not a promise that every long prompt sustains those rates. |
| 114.85 tok/s text; 104.25 tok/s GPU vision | A later matched text-versus-vision A/B reported by the repository | A separate comparison; do not merge these values with the baseline medians. |
These figures are not interchangeable averages. The repository notes that decode counts reasoning and answer tokens, runs stop at 512 tokens, first-token latency is not a completed-answer measurement, and the compared columns have different cache histories. Its matched A/B is more useful for comparing the two profiles than baseline columns with differing cache histories, but it remains a result from one community project rather than an independently reproduced standard.
Why a 125B model can run on a 24 GB card in this report
Qwen3.8-Flash-Next is a multimodal sparse mixture-of-experts model. Qwen and NVIDIA documentation describe 125 billion total parameters and 6 billion active per token, plus a 51-billion-parameter n-gram embedding table. “6B active” does not mean the entire model and its associated data fit uncompressed in 24 GB of GPU memory. The reported 4090 configuration uses a quantized model and a system with much more host memory than the card has video memory; the GPU name alone does not describe the setup.
The model combines Gated DeltaNet (GDN) and Qwen Sparse Attention (QSA): three of every four layers use GDN, with the remaining layer using QSA. That model-specific architecture makes compatible inference software and kernels important. The community result establishes that this project reports a working configuration; it does not establish that any engine, quantization, or default installation will reproduce it.
How the community 4090 route differs from NVIDIA’s documented recipes
NVIDIA’s reviewed deployment guides describe higher-memory Blackwell configurations, not a standard RTX 4090 path. The requirements below belong to the stated NVIDIA recipes and should not be read as universal minimums for every third-party quantized engine.
| Deployment path | Documented hardware or memory figure | What it means for a 4090 owner |
|---|---|---|
| NVIDIA Dynamo Qwen3.8-Flash-Next NVFP4 recipe | Approximately 130 GB minimum GPU memory; 51 GB host RAM per worker | This is a recipe-specific NVFP4 path, not a 24 GB RTX 4090 configuration. NVIDIA describes it as using an upstream vLLM image with GDN/QSA kernels. |
| NVIDIA TensorRT-LLM NVFP4 guide | One Blackwell GPU with 100 GB GPU memory and 96 GiB host memory | This is also a different hardware path from an RTX 4090. |
| Reported community configuration | One 24 GiB RTX 4090, Ryzen 7900, 192 GB RAM, unpruned GSQ IQ3_S | A separate, quantized community setup; do not substitute its requirements or results for NVIDIA’s recipes. |
NVIDIA’s RTX 4090 specification lists 24 GB of GDDR6X memory and 1,008 GB/s memory bandwidth. Those specifications do not by themselves show that the card can hold or run this model at a given speed. The reported community result depends on the quantization, host-memory setup, software path and profile in addition to the GPU.
What to check before attempting a local setup
The available benchmark description supports the reported hardware and performance figures, but it does not provide enough verified installation commands or a complete software configuration here to reproduce the run step by step. Avoid treating a generic model download or an NVIDIA Blackwell recipe as the missing RTX 4090 instructions. Before following a third-party setup, confirm that it identifies the exact model files, quantization, inference engine and compatible GDN/QSA kernels used.
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4. System Requirements: Minimum 850W PSU with 16-pin 12V-2x6 (12VHPWR) connector required. Verify before purchasing.
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability. Compatibility: 348mm (13.7") length, 3.6 slots, 4.3 lbs. Confirm case clearance and slot spacing. GPU bracket included.
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.6-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
- GPU and host memory: Verify available VRAM and system RAM against the chosen quantized build and engine. The cited community machine used 192 GB RAM; that is a description of its setup, not a proven universal minimum.
- Quantization: Check that the distribution is the unpruned GSQ IQ3_S configuration if you are trying to match the benchmark. A different quantization can change memory use, output quality and speed.
- Engine and kernels: Confirm support for this model’s GDN/QSA architecture and the selected quantization. A model loading successfully is not proof that it is using the same optimized path as the benchmark.
- Workload: Record whether vision is enabled, prompt/context length, reasoning mode, output length and cache state. The text and GPU-vision results differ, and the project warns that cache histories are not uniform across all columns.
- Measurement: Separate prefill, time to first token and decode throughput. Compare like with like: same profile, context, output length and run conditions.
Context length is not a speed promise
Qwen documentation gives the model a native context length of 262,144 tokens and says YaRN can extend it to 1 million tokens. These are context-capacity figures, not a guarantee that a local 4090 setup can serve the full context at benchmark decode rates. Longer contexts and different workloads affect memory demand and performance.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →The Qwen model card advises changing YaRN configuration for long-context use and warns that static YaRN can affect shorter texts. Apply such a configuration only when the intended context requires it; do not assume that enabling the maximum extension is beneficial for ordinary prompts.
How to judge another “100 tok/s” claim
A useful comparison needs more than a GPU model and one speed number. Check the conditions that make the result reproducible and meaningful:
- GPU model, memory capacity and architecture
- System RAM, CPU and whether model data is offloaded between GPU and host memory
- Exact quantization and whether the model is pruned
- Inference engine and model-specific kernel support
- Prompt length and context setting
- Text-only or vision-enabled profile
- Whether the quoted value is prefill, decode, first-token latency or aggregate serving throughput
- Number and length of runs, cache state, reasoning mode and any power limit
For scale, NVIDIA reported more than 16,000 tokens per second per GPU and more than 200 tokens per second per user on GB300 NVL72 in 2026. That is a rack-scale Blackwell result, not a comparable RTX 4090 measurement.
What a 4090 owner should take away
More than 100 generated tokens per second on an RTX 4090 is a reported outcome for one demanding, quantized community configuration—not a performance level guaranteed by the card or by NVIDIA’s official recipes. The most relevant published baseline is 110.8 tok/s median for the text profile and 97.65 tok/s for the GPU-vision profile, on a system with 192 GB RAM. Treat other numbers from the same project as profile- and context-specific measurements, and use the exact workload conditions when deciding whether a local result is comparable.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
