Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →GLiClass’s first native FP8 adapter was slower than BF16: Yuri Pocepaev measured a 59.23 ms median for FP8 versus 24.18 ms for eager BF16 on an RTX 4050 Laptop GPU. After fusing auxiliary work with Triton and capturing encoder execution in CUDA Graphs, his FP8 median fell to 16.10 ms. That is a 3.68× improvement over his initial FP8 implementation—not a general claim that FP8 is always faster.
What the latency numbers compare
Pocepaev’s 2026 report measures one batch-1 request: a fixed AG News example classified against four candidate labels. Each timed request included tokenization and postprocessing. The author warmed up the model, evaluated quality, then timed 50 additional requests, synchronizing CUDA before and after each timing. Model loading, Triton compilation, and CUDA Graph preparation were excluded; laptop clocks and thermals were not locked.
As an Amazon Associate I earn from qualifying purchases.
| Runtime | Median latency | p95 latency | Peak allocated tensor memory |
|---|---|---|---|
| BF16, eager | 24.18 ms | 29.67 ms | 3.219 GiB |
| BF16 + CUDA Graphs | 23.79 ms | 24.45 ms | 3.238 GiB |
| Initial native FP8 adapter | 59.23 ms | 66.36 ms | 2.145 GiB |
| FP8 + Triton | 37.97 ms | 46.62 ms | 2.144 GiB |
| FP8 + Triton + CUDA Graphs | 16.10 ms | 16.44 ms | 2.166 GiB |
These are the author’s measurements from one setup, not an independently reproduced benchmark or a multi-system test. Against the same-graph BF16 control, optimized FP8 was 1.48× faster on this request. The 3.68× figure instead compares two FP8 adapter versions. The table’s memory column is allocated tensor memory during the measured run, not total GPU memory use.
Why the first FP8 version was slower
FP8 covered selected projections, not every tensor
The derivative checkpoint quantizes 168 matrices across 24 mT5 encoder blocks. It stores weights in FP8 E4M3 with per-output-channel scales and dynamically quantizes activations. Embeddings, normalization layers, and classification components remain BF16, so W8A8 describes selected projection operations rather than the whole model.
#1 Best Overall
- ️ [PROCESSOR] Reinforced with Intel Core i5 13420H processor, up to 4.6GHz with Intel Turbo Boost technology, 12MB cache and 8 cores
- ️ [GRAFIIC] NVIDIA GeForce RTX 4050 GPU Fast Graphics for Laptops (GDDR6 6GB) to get more FPS in all your matches stably
- 16GB DDR4 RAM memory.
- ️ [STORAGE] Enjoy your favorite apps 512GB NVMe PCIe SSD drives
- ️ [SCREEN] 15.6 inch 144 Hz full HD display (1920 x 1080) with micro edges and anti-glare to make the screen as comfortable as possible.
Pocepaev reports weight-file sizes of 2,259,902,516 bytes for the derivative checkpoint and 3,416,522,340 bytes for BF16, a 33.85% reduction. Smaller weights saved storage, but did not by themselves make inference quicker.
Quantization and dispatch overhead erased the expected gain
The initial adapter used torch._scaled_mm with cuBLAS for the FP8 matrix multiplications. Profiling still counted 3,269 GPU kernel executions for the request, including separate work to find activation maxima, calculate scales, convert types, pad rows, and process outputs. Those auxiliary operations and launches outweighed the benefit of using smaller-precision matrix multiplications in this implementation.
Rank #2
- 【RAM & Storage】This computer comes with 16GB RAM DDR5 | 512GB SSD
- 【AMD Ryzen 7 7435HS Processor】It is a mid-range CPU for gaming laptops that features 8 Zen 3+ cores and no iGPU.This 16-thread Rembrandt Refresh family processor runs at 3.1 GHz to 4.5 GHz.
- 【15.6-inch FHD display with 144 Hz refresh rate】1920 x 1080 resolution draws you into crisp gameplay. Never miss a cutscene again with the 144Hz refresh rate that reduces frustrating lag and image ghosting. AMD FreeSync Premium technology delivers smooth gameplay with a high refresh rate, low framerate compensation, and low latency. IPS (in-plane switching), micro-edge, anti-glare display in sRGB color space.
- 【Other features】NVIDIA GeForce RTX 4050,Front-Facing Camera,Built-In Microphone,DTS: X Ultra Technology，AI Noise reduction， Windows 11 Home OS.
- 【Bundle with mouse pad】Bundled with Mouse Pad featuring high-quality cloth surface promotes smooth mouse gliding and enhanced precision.
What changed to reach 16.10 ms
Triton fused work around the matrix multiplications
The first optimization moved activation quantization into a Triton kernel and fused output scaling into a second Triton kernel. Median latency dropped from 59.23 ms to 37.97 ms. This step reduced auxiliary overhead while retaining native FP8 GEMMs.
Recommended Free Tools
CUDA Graphs reduced encoder dispatch overhead
The next step captured encoder work in CUDA Graphs. The adapter uses sequence-length buckets of 64, 128, 192, and 256 tokens, padding an input to its bucket and masking extra positions. One graph-capture compatibility issue was handled by constructing the attention mask outside the captured region.
Rank #3
- POWERFUL PERFORMANCE: AMD Ryzen 5 7235HS processor with 16GB memory delivers smooth multitasking and responsive gaming performance for demanding applications
- IMMERSIVE GRAPHICS: NVIDIA GeForce RTX 4050 graphics card provides exceptional visual quality with ray tracing capabilities for modern gaming titles
- DISPLAY: 15.6-inch Full HD screen offers crisp 1920x1080 resolution for detailed visuals and an immersive gaming and entertainment experience
- FAST STORAGE: 512GB solid-state drive provides ample space for games and files with quick boot times and rapid data access speeds
- SLEEK DESIGN: Luna Grey finish with portable gaming laptop form factor makes it easy to game at home or on the go
With Triton fusion and graphs together, the profiled request used 1,425 GPU kernel executions rather than 3,269 for the initial adapter; all 168 native FP8 GEMMs remained. The resulting 16.10 ms median is specific to this implementation and measured request shape.
What the quality check does—and does not—show
The report compares BF16 and optimized FP8 on 664 examples: a seeded subset of 256 AG News test examples, plus the full English and Russian SIB-200 test splits of 204 examples each. SIB-200 candidate labels were in English for both language splits.
Rank #4
- Beyond Performance: The Intel Core i5-13420H processor goes beyond performance to let your PC do even more at once. With a first-of-its-kind design, you get the performance you need to play, record and stream games with high FPS and effortlessly switch to heavy multitasking workloads like video, music and photo editing.
- AI-Powered Graphics: The state-of-the-art GeForce RTX 4050 graphics (194 AI TOPS) provide stunning visuals and exceptional performance. DLSS 3.5 enhances ray tracing quality using AI, elevating your gaming experience with increased beauty, immersion, and realism.
- Visual Excellence: See your digital conquests unfold in vibrant Full HD on a 15.6" screen, perfectly timed at a quick 165Hz refresh rate and a wide 16:9 aspect ratio providing 82.64% screen-to-body ratio. Now you can land those reflexive shots with pinpoint accuracy and minimal ghosting. It's like having a portal to the gaming universe right on your lap.
- Internal Specifications: 8GB DDR5 Memory (2 DDR5 Slots Total, Maximum 32GB); 512GB PCIe Gen 4 SSD
- Stay Connected: Your gaming sanctuary is wherever you are. On the couch? Settle in with fast and stable Wi-Fi 6. Gaming cafe? Get an edge online with Killer Ethernet E2600 Gigabit Ethernet. No matter your location, Nitro V 15 ensures you're always in the driver's seat. With the powerful Thunderbolt 4 port, you have the trifecta of power charging and data transfer with bidirectional movement and video display in one interface.
| Evaluation set and measure | BF16 | Optimized FP8 | Difference |
|---|---|---|---|
| AG News macro-F1 | 79.08% | 79.49% | +0.41 percentage points |
| SIB-200 English macro-F1 | 84.57% | 84.04% | −0.53 percentage points |
| SIB-200 Russian macro-F1 | 84.09% | 83.42% | −0.67 percentage points |
| Top-1 prediction agreement across all 664 examples | BF16 and optimized FP8 agreed on 99.25% | Agreement is not accuracy | |
The small AG News increase is not evidence that quantization improved the model, and the SIB-200 scores show small declines. This evaluation does not establish quality retention across other languages or production tasks. The report also does not preserve full score distributions, so it makes no claim about all-logit changes or calibration.
How to interpret or reproduce the result
The result is a workload-specific inference engineering report, not a general GPU benchmark. A meaningful comparison with another implementation should align the details that affect latency and quality:
Best Value
- READY FOR ANYTHING – Dive headfirst into gaming on Windows 11 powered by the Intel Core i5 Processor 13450HX and an NVIDIA GeForce RTX 5050 Laptop GPU with a Max TGP of 115W and NVIDIA Advanced Optimus.
- SUBTLE STYLING – The TUF Gaming F16 maintains its classic design, boasting a subtle embossed TUF logo on its sleek cover.
- IMMERSIVE VISUALS – The TUF Gaming F16’s FHD+ 165Hz display with 100% sRGB color draws you into the action. Adaptive-Sync technology reduces lag, minimizes stuttering, and eliminates visual tearing for ultra-smooth gameplay.
- MILITARY GRADE DURABILITY – As a TUF gaming machine, the F16 has been rigorously tested to meet Military Grade testing standards, MIL-STD-810H. Rest easy knowing this laptop will operate at peak performance in harsh conditions.
- EFFICIENT COOLING – Equipped with 2nd Gen Arc Flow Fans, full-width heatsink, and full-width vent, the TUF Gaming F16 optimizes cooling performance without extra noise.
- GPU model and laptop power or thermal conditions; the reported clocks and thermals were not locked.
- Model and checkpoint revision, software stack, batch size, sequence length, and number of candidate labels.
- Warmup and synchronization method, and whether tokenization and postprocessing are part of the timed request.
- Whether loading, compilation, and graph preparation are included, and whether quality is evaluated on the same examples.
The report does not measure larger-batch sustained throughput, other GPUs, other operating systems, or production workloads. Its roughly 2.17 GiB optimized-run allocation is not a full loading-memory requirement: the author says the loader temporarily reconstructs weights in BF16 before replacing projections. Nor is allocated tensor memory equivalent to total VRAM use reported by nvidia-smi.
Pocepaev describes a Hugging Face repository with the weights, tokenizer, dependencies, adapter, and an inference.py entry point, and says the model, code, and report are available under Apache-2.0. The current repository URL and artifact revision are not established here, so verify the repository and revision before relying on those availability or licensing details.
The engineering lesson
As Pocepaev puts it, “The main takeaway is simple: quantization gives you a smaller representation; making it faster is a separate engineering problem.” In this case, FP8 reduced weight-file size, while Triton fusion and CUDA Graphs addressed the extra work and dispatch overhead that had made the first FP8 path slow.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchQuick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
