The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Yes, a 128 GB Mac can run the full 4-bit MLX build of Qwen3.8-Flash-Next at long context, but not with default settings. In a report published September 24, 2026, Nariaki Wada says the full build failed on his Mac Studio M4 Max (128 GB) until he stopped loading the model’s huge n-gram embedding table into ordinary resident memory. Memory-mapping that table and reading only the rows each request needs let the full model handle prompts up to 240K tokens. The alternative he tried first, an expert-pruned build, fit more easily but lost quality in his Japanese and general-knowledge evaluation.
Everything below about the Mac run (memory, timings, quality) comes from that one author’s setup and tests. I have not seen an independent replication, so treat the numbers as a well-documented single data point, not a specification.
As an Amazon Associate I earn from qualifying purchases.
Why a “125B” model is awkward on a 128 GB machine
The Qwen Team’s architecture paper (arXiv:2608.30320, 2026) describes Qwen3.8-Flash-Next as a sparse mixture-of-experts model with 125B total parameters and roughly 6B active per token. On top of that backbone sits a 51B-parameter n-gram embedding table. The paper’s own wording is that “capacity is added outside the backbone by a single n-gram embedding layer whose tables are prefetched from host memory” (Qwen Team, 2026).
Two consequences matter on a Mac:
- Active parameters are not memory. Only ~6B parameters do work for each token, which is why generation can be fast. But all experts, and the n-gram table, still have to be reachable. Neither the 125B nor the 6B figure by itself tells you how much memory a particular quantized checkpoint needs.
- The n-gram table is designed to live off the accelerator. The architecture expects it in host memory and fetched on demand. On Apple Silicon, where CPU and GPU share one memory pool, “off the accelerator” doesn’t automatically mean “out of the way”, unless the runtime is told to treat it that way.
The paper corroborates the model’s design. It does not benchmark Macs, so it can’t confirm or refute the Mac results below.
#1 Best Overall
- BRAWN OF A NEW AGE — Mac Studio is a tremendously powerful pro desktop. The M5 Max chip enables remarkable on-device AI compute. Blast through creative projects and professional workflows with the advanced graphics architecture and faster memory and storage.
- M5 MAX CHIP — Tap into breakthrough performance with a next-generation CPU, a more powerful GPU with third-generation ray tracing, and a Neural Accelerator built into each GPU core. Mac Studio gets a boost with more power to generate real-time media and accelerate complex workflows.
- MEMORY AND STORAGE — Get up to 128GB unified memory and up to 614GB/s memory bandwidth for more speed when processing massive datasets, complex 3D scenes, and inference in AI workflows. And up to 2x faster storage* expedites tasks like file transfers and loading large projects.
- A POWERFUL PLATFORM FOR AI — Apple silicon, and every major component that powers it, is designed to run demanding AI workflows like running huge LLMs, directly on device. And Apple Intelligence* helps you write, express yourself, and get things done effortlessly, while Siri AI* is your profoundly capable assistant — all with groundbreaking privacy protections.
- A POWERFUL PLATFORM FOR AI — Apple silicon, and every major component that powers it, is designed to run demanding AI workflows like running huge LLMs, directly on device.
What went wrong with the full 4-bit build
Wada’s account of the full 4-bit MLX build, on a Mac Studio M4 Max with 128 GB:
- Peak memory after loading reached 111.5 GB. On a 128 GB machine that leaves little headroom for the KV cache and prompt processing, which grow with context length.
- With default settings, a 32K-token retrieval task failed.
- Lowering the prefill step size got 32K through, but 128K still did not work.
These are the author’s observations on one machine and one build. They are not a general minimum-memory figure for the model.
The expert-pruning detour
When a model won’t fit, removing experts is the obvious lever. The author tested an expert-pruned variant (REAP-288). It fit more easily, but in his own evaluation it showed losses in Japanese and in general knowledge. The full 4-bit build kept the best quality of what he tried.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Rank #2
- UNMATCHED PERFORMANCE - Experience blazing-fast speeds with the M3 Ultra or M4 Max chip, featuring up to a 32-core CPU and up to 80-core GPU for demanding tasks like video editing and 3D rendering.
- STUNNING VISUALS - Connect up to eight displays with the M3 Ultra, or five displays with the M4 Max, supporting resolutions up to 8K for immersive and highly detailed visual experiences across multiple screens.
- AMPLE MEMORY - Configure up to 512GB of memory with the M3 Ultra or up to 128GB with the M4 Max, ensuring smooth multitasking and efficient handling of large datasets for professional workflows.
- EXPANSIVE STORAGE - Choose from a range of SSD options, from 512GB to a massive 16TB, providing lightning-fast access to your files and ample space for all your creative projects and important data.
- VERSATILE CONNECTIVITY - Equipped with Thunderbolt 5 ports delivering up to 120Gb/s, USB 3 ports, HDMI 2.1, and 10Gb Ethernet, this desktop offers seamless integration with all your peripherals and networks.
How far to generalize that depends on what you do with the model:
- Your language and task mix decides whether pruning hurts. Wada’s losses showed up in Japanese and general knowledge. A pruned build may look fine on one task family and degrade on another.
- Pruning and quantization are separate trade-offs. Removing experts and lowering bit width each cost something, and a comparison that changes both at once can’t tell you which one caused a regression. One community variant, sh0wie’s “REAP-288 Q8E” (8-bit experts on a 4-bit backbone), is a good example of how these choices stack. Its Hugging Face model card reports its own HumanEval figures and cautions that its results are tied to that specific build.
- Don’t swap benchmarks. The model card’s code results say nothing about Japanese or general knowledge, and Wada’s evaluation says nothing about HumanEval. Neither substitutes for testing on your own prompts.
The practical lesson is not that pruning is always bad. It is that pruning solved a memory problem by changing the model, when the memory problem turned out to have a cheaper cause.
The actual fix: stop holding the n-gram table in resident memory
Wada found that the converted model lacked ple-store.json, a manifest used by the external per-layer-embedding (PLE) storage path in mlx-vlm. Without it, the large table was loaded as ordinary MLX parameters, which is a big part of the 111.5 GB peak.
Rank #3
- Take on extreme workflows, from detailed visual effects and 3D animation to film scoring. The powerful Neural Engine supports AI assistance in complex tasks, and the advanced GPU architecture supports Dynamic Caching, mesh shading, and ray tracing
- Phenomenal memory and storage - Get up to 128GB unified memory and up to 8TB storage with M4 Max or up to 512GB unified memory and up to 16TB storage with M3 Ultra
- Built for apple intelligence - Apple Intelligence is the personal intelligence system that helps you write, express yourself, and get things done effortlessly. With groundbreaking privacy protections, it gives you peace of mind that no one else can access your data-not even Apple
- Fits right on your desk - The compact 7.7" square Mac Studio enclosure is designed to fit perfectly under most displays
- Runs cool and quiet - The thermal system is designed to let you fly through intensive tasks at incredible speeds while keeping Mac Studio quiet, so it never interferes with your workflow
With the external-storage setup in place, the table is kept in a memory-mapped file and only the rows a request actually needs are read from storage. The full build, with no experts removed, then ran prompts up to 240K tokens on the same Mac.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsHow to check whether your own setup has this problem
- Look in your converted model directory for
ple-store.json. If it is missing and your build has a large PLE/n-gram table, you are probably loading that table as regular parameters. - Follow the current mlx-vlm instructions for external PLE storage to generate the store and manifest. The exact commands and flags depend on your mlx-vlm version, so use the project’s current documentation rather than a copy from an older post.
- Reload the model and compare peak memory after loading against the 111.5 GB figure Wada saw without it.
- Re-run a retrieval-style test at 32K, then step upward. If a larger context fails, adjust prefill step size as he did at 32K, and treat that as a separate knob from the table fix.
Expect your results to differ. Storage speed, free disk space, macOS version, and library versions all affect a method that reads from disk during inference. Wada’s report does not give a storage model or throughput, so I can’t tell you what drive you need. Sizing the disk is simple: it must hold the model artifacts you select, including the external table.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What the 240K run looked like
Wada tested prompts up to about 240K tokens and compared Flash-Next with memory-mapped PLE against Qwen3.8-27B. Each time is the total through a roughly 50-token answer, so it is dominated by prompt processing, not generation.
Rank #4
- Powerful M5 Max Performance – Apple MacBook Pro 16-inch with M5 Max chip, featuring an 18-core CPU and 40-core GPU for demanding creative workflows, multitasking, coding, editing, and professional productivity.
- 128GB Unified Memory – Built with 128GB unified memory to help handle large files, complex projects, multiple pro apps, and heavy workloads with smooth, responsive performance.
- Fast 2TB SSD Storage – The 2TB solid-state drive provides fast file access, quick app launching, and spacious storage for videos, photos, documents, software, media libraries, and professional projects.
- 16-Inch MacBook Pro Display – Designed with a large 16-inch display for sharp detail, rich color, and a premium viewing experience for creative work, business tasks, entertainment, and everyday use.
- Professional Laptop Configuration – High-performance MacBook Pro setup built for creators, designers, developers, photographers, video editors, business users, and power users who need advanced speed and capability.
| Model (author’s Mac Studio M4 Max, 128 GB) | Total time at ~240K-token prompt |
|---|---|
| Qwen3.8-Flash-Next, full 4-bit, memory-mapped PLE | 447.0 s (7.5 min) |
| Qwen3.8-27B | 2,114.9 s (35.2 min) |
That is about 4.7 times faster for the sparse model in this test. It reflects the author’s stated conditions and does not establish a universal speedup for other prompts, models, or machines. It is plausible given the architecture, since only about 6B parameters are active per token, but a plausible mechanism is not a replication.
The same report’s claim that memory-mapping lets the full model run without quality loss is also a comparison within his experiment. It is not independent or standardized quality proof.
Boundaries you shouldn’t blur
- 240K tested is not the context limit. The architecture’s stated native context is 262,144 tokens. Wada tested up to 240K; he did not report beyond that, and none of the sources check extended-context behavior on a Mac.
- The wired-memory limit is untested. The author explicitly says he did not try raising macOS’s GPU wired-memory limit (
iogpu.wired_limit_mb). It may or may not help; it is not a verified fix here. - The Mac method is not the vLLM method. The vLLM project’s deployment recipe documents PLE CPU offload on CUDA/ROCm systems with hardware-specific configurations, and says the documented offload currently runs on NVIDIA devices. That is a different implementation from mlx-vlm’s memory map, so don’t carry settings or expectations between them.
- Software moves. mlx-vlm, vLLM, and conversion tooling change quickly. Re-check the current instructions before following any step above.
Is the paper’s efficiency claim relevant to your Mac?
Partly. The Qwen Team reports that on fourteen pre-training benchmarks, the model leads its 397B-A17B predecessor on eight and trails on the other six by at most 2.6 points, while using roughly one third of the activated parameters, one third of the training tokens, and about one ninth of the training FLOPs. That is pre-training evidence from the model’s authors. It explains why a model with 6B active parameters is worth the trouble of fitting, but it is not a consumer benchmark and says nothing about quantized or pruned builds on a Mac.
Choosing between full and pruned builds
| Question | Full 4-bit + memory-mapped PLE | Expert-pruned (e.g. REAP-288) |
|---|---|---|
| Quality in Wada’s Japanese and general-knowledge evaluation | Best of the builds he tried | Losses reported |
| Memory pressure | Reduced by keeping the table out of resident memory; 111.5 GB peak was the unmapped case | Fits more easily |
| Long context on a 128 GB Mac | Reported up to ~240K tokens | Not stated in the report |
| Extra requirement | External PLE store, ple-store.json, and storage reads |
None beyond the pruned checkpoint |
| Evidence base | One author’s tests | One author’s tests plus a model card with its own HumanEval figures, tied to that build |
Start with the full build and fix the table loading. Reach for a pruned model only if you’ve tested it on your own languages and tasks and found the loss acceptable.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
