Short answer: Nemotron 3.5 Super VL is not a conventional single-card model in BF16. NVIDIA describes it as a 120-billion-parameter total, 12-billion-active hybrid Mamba-Transformer mixture-of-experts vision-language model, and its published BF16 profiles require two, four or eight GPUs depending on the hardware. The lowest published BF16 floor is 32 GB per GPU on an eight-way H100 80GB profile, while a supported NVFP4 profile can start at 72 GB on one high-memory GPU or 38 GB on each of two GPUs.
Those figures are startup floors for NVIDIA-verified profiles, not universal workload guarantees. Long prompts, images, video, conversation history, tool results and concurrent requests consume KV-cache memory on top of the model and runtime allocation. The reliable way to prevent OOM errors is to match the exact NVIDIA profile to your GPU, choose BF16 or NVFP4 deliberately, leave utilization headroom, and size context, concurrency, media and host RAM together.
As an Amazon Associate I earn from qualifying purchases.
What Nemotron 3.5 Super VL is loading into memory
Nemotron 3.5 Super VL accepts text, images and video and returns text. NVIDIA describes the architecture as interleaved Mamba-2, mixture-of-experts and attention layers, with a C-RADIO vision encoder and a vision-language connector. Its advertised context window reaches 1 million tokens. That window is shared by the user prompt, image and video representations, chat history, tool output and generated text; it is not a separate allowance for each input type.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
The model has 120 billion total parameters but 12 billion active parameters in the MoE computation. “Active” does not mean that only 12 billion parameters need to be stored when the service starts. The published deployment layouts account for the complete model, runtime allocations and tensor parallelism. NVIDIA’s model card lists deployment on four H100 GPUs or two GB200 GPUs, while the NIM support matrix provides additional verified layouts.
#1 Best Overall
- AI Performance: 767 AI TOPS
- OC mode: 2632 MHz (OC mode)/ 2602 MHz (Default mode)
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Axial-tech fan design features a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- A 2.5-slot design maximizes compatibility and cooling efficiency for superior performance in small chassis
Published VRAM floors and GPU layouts
The following numbers are NVIDIA’s profile-specific minimum memory values at startup. They include model weights and runtime allocations, but exclude the extra KV-cache headroom needed for long context or high concurrency.
| Precision/profile | Published layout | Minimum memory floor | What it means |
|---|---|---|---|
| BF16, TP2 | 2 × B200, B300, GB200 or GB300 | 119 GB per GPU | A high-memory Blackwell or Grace Blackwell system; reserve additional memory for KV cache and requests. |
| BF16, TP4 | 4 × H200 | 61 GB per GPU | A four-GPU H200 deployment whose usable context and concurrency depend on remaining headroom. |
| BF16, TP8 | 8 × H100 80GB | 32 GB per GPU | The verified H100 profile uses eight GPUs; it is not evidence that one 32 GB card can run the model. |
| NVFP4, TP1 | 1 × B200, B300, GB10, GB200, GB300 or H200 | 72 GB | Quantized weights can fit on one supported high-memory GPU, subject to KV-cache, media and runtime demand. |
| NVFP4, TP2 | 2 × A100 80GB, H100 80GB or RTX PRO 6000 Blackwell Server Edition | 38 GB per GPU | A verified two-GPU quantized layout with lower per-card requirements than BF16. |
For example, 32 GB is a per-GPU startup floor for the eight-H100 BF16 profile, not a 32 GB total requirement. Likewise, 38 GB, 61 GB, 72 GB and 119 GB should not be read as guarantees for every context length, batch size, media mix or runtime. A service can pass startup and still run out of memory when traffic increases.
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5070 Ti
- Integrated with 16GB GDDR7 256bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
Can it run on one GPU?
BF16: not in the published profiles
NVIDIA’s published BF16 configurations are tensor-parallel multi-GPU deployments: two B200/B300/GB200/GB300 GPUs, four H200 GPUs or eight H100 80GB GPUs. A single H100, H200 or similar card is therefore not a supported BF16 layout in those profiles. Adding an arbitrary number of cards also does not automatically create a supported configuration; the model’s profile selection is tied to GPU model and count.
NVFP4: one GPU can be supported
NVFP4 is NVIDIA’s lower-memory option for supported hardware. The TP1 profile lists one B200, B300, GB10, GB200, GB300 or H200 with a 72 GB minimum. That makes one-GPU inference possible on the listed hardware, but it does not make the model “small”: a long context, multiple sequences or image and video decoding can consume the remaining memory. A card that merely has 72 GB should be treated as a starting point, not a promise of 1-million-token operation.
Rank #3
- Powered by the NVIDIA Blackwell architecture and DLSS 4. System Requirements: Minimum 850W PSU with 16-pin 12V-2x6 (12VHPWR) connector required. Verify before purchasing.
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability. Compatibility: 348mm (13.7") length, 3.6 slots, 4.3 lbs. Confirm case clearance and slot spacing. GPU bracket included.
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.6-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
BF16 versus NVFP4: choose for the workload
Use BF16 when you need the behavior and workflow associated with a higher-precision deployment, such as customization, LoRA work or full-precision evaluation. Use the official NVFP4 release and profile when quantized inference is acceptable and your GPU appears in a supported NVFP4 layout. The choice also determines how many GPUs you need and how much memory remains for requests.
| Decision factor | BF16 | NVFP4 |
|---|---|---|
| Published scale | TP2, TP4 or TP8 layouts, depending on GPU family | TP1 or TP2 layouts on listed GPUs |
| Lowest published per-GPU floor | 32 GB, but only in the eight-GPU H100 profile | 38 GB in verified two-GPU profiles; 72 GB for verified one-GPU profiles |
| Customization and LoRA | The recommended direction when these are required | Use only when your quantized workflow is acceptable and supported |
| Evaluation | Appropriate for full-precision evaluation requirements | Evaluate quality and tooling against your quantized objective |
| Profile requirement | Must match GPU family and tensor-parallel count | Must match the exact NVFP4 hardware profile |
Why Nemotron runs out of memory after apparently fitting
KV cache expands with context and concurrency
Weights are only the first allocation. The KV cache stores attention state for active sequences, so its size rises with the number of tokens retained and the number of sequences being served. A 1-million-token maximum is a capability, not a sensible default for every GPU layout. A coding agent that preserves a long history, tool calls and generated output can use far more cache than a short single-turn question.
Rank #4
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5060
- Integrated with 8GB GDDR7 128bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
Two requests with the same model can therefore have very different memory footprints. Lowering the maximum context, reducing concurrent sequences, trimming old tool traces and limiting output tokens are the main controls when startup succeeds but requests later fail.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsImages and video have their own reservation
Except on DGX Spark, NIM reserves 10 GiB of GPU memory for image and video decoding. Media requests wait for that reservation rather than immediately colliding with other allocations. The reservation protects decoding, but it also reduces memory available for model execution. Reducing the reservation can leave more room for KV cache while making large or simultaneous media requests wait or serialize sooner.
Best Value
- Powered by the NVIDIA Blackwell architecture and DLSS 4 OC mode: 2640MHz/Default mode: 2610MHz (Boost Clock)
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.125-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
Agent workloads can exhaust system RAM
NIM agentX profiles offload KV cache to CPU memory. They require 200 GiB of host memory per GPU, or 400 GiB for the two-GPU H100 agentX profile, in addition to the memory used by the operating system, containers and other services. A machine can consequently fail from host-memory pressure even while its GPU usage looks acceptable.
An automatically selected profile may be wrong
For this release, NVIDIA says automatic selection does not filter profiles by GPU model or per-GPU memory. An unsuitable profile can fail during startup or request an allocation your cards cannot satisfy. Explicit profile selection is safer than assuming the first listed option is compatible.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.A deployment process that avoids preventable OOMs
- Inventory the hardware. Record the exact GPU model, memory per card, GPU count, interconnect and available system RAM. Distinguish an RTX PRO 6000 Blackwell Server Edition from similarly named workstation products; only the server edition appears in the cited two-GPU NVFP4 profile.
- Choose precision. Select BF16 for workflows that require higher-precision customization, LoRA or evaluation. Select NVIDIA’s NVFP4 profile when quantized inference is suitable and the hardware is explicitly listed.
- Match tensor parallelism. Use the published TP2, TP4, TP8 or TP1 layout that corresponds to your GPU family. Do not infer support by dividing the total parameter count by an arbitrary number of cards.
- Inspect available profiles. Run the deployment’s
list-model-profilescommand and read the profile names, GPU requirements and precision. Select the profile that matches both hardware and objective. - Set the profile explicitly. Configure
NIM_MODEL_PROFILEto the selected profile name or ID instead of relying on automatic selection. - Start with headroom. NVIDIA’s documented BF16 vLLM example uses
--gpu-memory-utilization=0.85. Treat 0.85 as a documented starting point, not a universal safe value; lower it when runtime allocations or cache growth approach the limit. - Set a realistic context. Begin below the 1-million-token maximum and increase only after measuring the workload. Include image and video tokens, retained history, tool results and expected output in the budget.
- Limit concurrency. Start with one or a small number of sequences, then raise concurrency while watching per-GPU memory and latency. A profile that works for one short prompt may not support many long sessions.
- Load-test the real media mix. Test representative image counts, resolutions, video frame counts and simultaneous requests. Text-only tests do not expose decoding reservations or media-related pressure.
- Check host RAM for agentX. Provision the documented 200 GiB per GPU, or 400 GiB for the two-GPU H100 agentX profile, before enabling CPU KV-cache offload.
Practical NIM and vLLM tuning
When startup fails
- Confirm that the selected profile names the actual GPU family and GPU count.
- Switch from an unsupported BF16 layout to the exact published layout, or use a supported NVFP4 profile if quantization is acceptable.
- Lower
--gpu-memory-utilizationfrom the documented 0.85 starting point when framework or runtime allocations leave no margin. - Make sure model-cache storage is not being confused with VRAM. Disk space for cached model files does not prove that weights can be loaded into GPU memory.
When only long requests fail
- Reduce maximum context length before changing the model profile.
- Lower the number of concurrent sequences.
- Trim old conversation turns, tool output and retrieved documents.
- Cap generated tokens for tasks that do not require lengthy answers.
When media requests fail or stall
- Reduce image and video count, resolution or frame sampling.
- Keep the 10 GiB decoding reservation in mind on non-DGX-Spark systems.
- If you lower the media reservation, expect large or concurrent media requests to queue more often.
When the GPU is fine but the process dies
- Inspect host-memory use, especially with agentX enabled.
- Budget the documented 200 GiB per GPU, or 400 GiB for the two-GPU H100 profile, plus container and operating-system needs.
- Reduce concurrent agent sessions or disable CPU KV-cache offload if the machine cannot supply that RAM.
Plan memory around the request, not just the model
A useful capacity plan has four separate lines: model and runtime allocation, KV cache, media decoding and host-memory offload. The published floor covers the first line at startup. The other lines vary with traffic. For a short text-only service, a verified profile may leave substantial room. For a multimodal agent retaining hundreds of thousands of tokens, the same profile can be unsuitable even though startup succeeds.
Recommended Free Tools
Document the intended maximum context, concurrent sequences, average and worst-case image or video inputs, output-token cap and whether agentX is enabled. Then test the worst combination, not merely an isolated prompt. If the service must support occasional long contexts, reserve capacity for those requests instead of running the GPUs at the startup floor.
Pre-launch checklist
- Is the GPU model and count an exact match for an NVIDIA-verified profile?
- Have you chosen BF16 or NVFP4 based on customization and quality requirements?
- Is
NIM_MODEL_PROFILEset explicitly? - Does the startup configuration leave utilization headroom rather than consuming every free byte?
- Are context length, concurrent sequences and output tokens bounded?
- Have media reservations and realistic image/video traffic been tested?
- If agentX is enabled, is the required host RAM available in addition to GPU memory?
- Are you treating the published floor as a startup minimum rather than a workload guarantee?
What the numbers do—and do not—guarantee
NVIDIA does not publish one VRAM number that guarantees success for every runtime, context length, batch size, concurrency level and image/video workload. The 32 GB, 38 GB, 61 GB, 72 GB and 119 GB figures are tied to named profiles and startup conditions. A reliable deployment combines the matching profile with conservative context and concurrency limits, media-aware testing and enough host RAM for any KV-cache offload.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
