For one NVIDIA DGX Spark, start with NVIDIA’s current single-device vLLM recipe: it targets the GB10 GPU and recommends Qwen3.8-27B NVFP4. Use the launch tab for the matching device configuration as the source of the full model, container, and serving settings; the available recipe description does not expose the complete command, so an older generic command should not be substituted for it. Two issues matter most: CPU and GPU workloads share unified memory, and GB10 low-bit support can depend on the vLLM build and kernels. Published token rates are meaningful only alongside their model and workload details.
Which vLLM recipe should you use on one DGX Spark?
NVIDIA’s vLLM recipe selector asks you to choose the matching device configuration. For “One DGX Spark,” it recommends Qwen3.8-27B NVFP4, a recipe intended for one GB10 GPU. NVIDIA says, “The launch tabs provide the complete model, container, and serving configuration.” Open the launch tab for that one-device option and use its generated settings together; the page’s descriptive text alone does not provide a complete command.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
NVIDIA RTX A400 4GB ATX | $369.00 | Buy on Amazon |
| 2 |
|
Vertical Stand Compatible with NVIDIA DGX Spark Desktop Computer Holder | $23.99 | Buy on Amazon |
As an Amazon Associate I earn from qualifying purchases.
This distinction matters because NVIDIA warns that its copy-and-paste launch configurations were tested for the recommended recipes. A different model or quantization may need different model-download, container, environment, memory, parser, or parallelism settings. Keep the model and image versions with any command you save or share, and do not treat a command for another recipe as interchangeable.
Why is memory the first landmine?
DGX Spark’s CPU and GPU use the same unified memory pool. The vLLM team’s June 1, 2026 DGX Spark guide describes a 128 GB pool shared by the operating system, container runtime, serving process, model weights, and KV cache. As requests run, the KV cache can grow, so assigning nearly all available memory to model serving can leave too little room for the rest of the system.
#1 Best Overall
- 900-5G172-2260-000
In vLLM, --gpu-memory-utilization controls how much memory the engine targets. Follow the value in the current recipe rather than assuming one percentage is best for every model or workload. If you change models, context lengths, or concurrency, revisit the memory setting and leave headroom for system use and cache growth.
What should you set for concurrency?
Match concurrency to the job. The vLLM guide says --max-num-seqs should stay low because DGX Spark suits small-batch inference better than high-concurrency serving. A high-concurrency throughput figure therefore describes a different workload from one person generating one answer at a time. Increasing concurrency may raise aggregate output while changing per-request speed and memory pressure.
- For interactive, single-user use, begin with the low-concurrency configuration in NVIDIA’s one-Spark launch recipe.
- For a multi-request workload, adjust concurrency deliberately and monitor memory; do not assume a benchmark at a larger batch predicts single-request latency.
- When reporting a result, identify whether token rate is per request or aggregate across requests.
Does a GB10 vLLM build support every low-bit format equally?
Compatibility and performance depend on the exact vLLM release, container, model quantization, and kernel path. A December 2025 NVIDIA Developer Forum post reported that NVIDIA’s Docker image then lagged mainline vLLM and that MXFP4 was not using Blackwell FP4 features on Spark in vLLM. That is a dated community observation, not a statement of current official compatibility. Check the recipe’s version and date before applying it to a present-day deployment.
The vLLM guide also notes that kernel and performance improvements vary by model and release; current builds generally use CUDA graphs unless a deployment has a reason to disable them. This does not establish that NVFP4 or MXFP4 will be faster in every setup. Use the official recipe as the baseline, then confirm behavior against the precise model and build you intend to serve.
What measured DGX Spark token rates are available?
Conselara Labs reported a third-party single-node result on May 9, 2026, using vLLM 0.19.0 in the NVIDIA nvcr.io/nvidia/vllm:26.04-py3 container for gpt-oss-120b. Its configuration used TP=1, MXFP4 quantization, FP8 KV cache, Triton attention, the Marlin MoE backend, and a 128,000-token context. The report gives approximately 32–35 generated tokens per second with reasoning overhead included, and 57–60 tokens per second for pure decode. These are results for that model and configuration, not for NVIDIA’s Qwen3.8-27B NVFP4 recipe.
Rank #2
- VERTICAL DESKTOP PLACEMENT: Designed to hold Compatible with NVIDIA DGX Spark devices in a vertical position, creating a different layout option for desktop computing setups
- SPACE-SAVING WORKSTATION DESIGN: The vertical holder helps reduce the footprint of compact computing equipment, making more room available around your desk area
- STABLE DEVICE HOLDER: Provides a dedicated placement space for compatible AI computing equipment, helping users arrange devices neatly on desks, shelves, or workstations
- OPEN STRUCTURE DESIGN: The simple open-frame structure keeps the surrounding area accessible, making daily device operation and workspace organization convenient
- AI WORKSPACE ACCESSORY: Suitable for AI development areas, home offices, maker spaces, and technology workstations where organized equipment placement is preferred
The same lab separately reported a two-node Qwen3-235B-A22B GPTQ-Int4 setup: 17.0 tokens/s at batch 1, 24.1 aggregate tokens/s at batch 2, and 36.4 aggregate tokens/s at batch 4. It reported about 15 minutes from startup to first inference for that two-node setup. These figures are not single-Spark performance numbers.
A participant in a separate NVIDIA Developer Forum thread reported about 59 generated tokens per second for single-user GB10 MXFP4 generation and noted additional throughput headroom with concurrency. The excerpt does not provide enough benchmark detail to equate that community report with the lab result.
For reproducible comparisons, a token-rate number needs context: model and quantization, vLLM and container versions, runtime flags and backend, KV-cache dtype, context length, prompt and output lengths, batch or concurrency, and whether reasoning is included. Also distinguish time-to-first-token from decode speed and aggregate throughput. NVIDIA’s hardware peak-compute specification is not a vLLM token-generation benchmark, and the sources do not establish a universal ranking of vLLM versus another serving runtime across unlike workloads.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Which DGX Spark memory configuration does the recipe assume?
NVIDIA identifies DGX Spark as a GB10 Grace Blackwell system. Its product specifications list 128 GB of coherent unified memory and a 64 GB configuration available through participating OEM partners. The official recipe and the cited single-node benchmark are discussed for a 128 GB system; do not assume their model fit or results transfer unchanged to the 64 GB configuration.
NVIDIA also lists 273 GB/s memory bandwidth and up to 1 PFLOP of FP4 tensor performance. Those hardware specifications describe the system, not the token rate a particular vLLM model will achieve.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitches
