Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Sharing a GPU can mean assigning it to one model at a time, letting several models contend for it, or switching models inside a serving system. Those methods are not interchangeable. In Christopher Maher’s August 10, 2026 LLMKube tests, four active time-sliced tenants sharply reduced throughput; switching models through ModelPool took tens of seconds, while a faster router switch failed to use GPU offload in the tested configuration. The corrected memory result also matters: a GPU-offloaded copy required roughly a full model’s size per instance, including on the tested unified-memory system.
Four ways to share a GPU operate at different layers
Maher’s report describes ten scenarios across two DGX Spark systems, using LLMKube’s supported sharing modes and additional model-management approaches. The four tested approaches decide who can use the GPU in different places: the scheduler, the operator, or the inference runtime.
As an Amazon Associate I earn from qualifying purchases.
| Approach | Where it acts | What happens | Best fit in the tested setup |
|---|---|---|---|
| Exclusive | Scheduler | One model is assigned the GPU; it does not share the card with another model. | One model or workload at a time. |
| Shared (time-slicing) | Scheduler | Multiple model workloads can be resident and take turns using the GPU. They compete when active at the same time. | Concurrent demand when memory is sufficient and workload slowdowns are acceptable. |
| ModelPool | Operator | One member is resident at a time. Moving to another member replaces the pod and loads its model. | Workloads that alternate between models and can tolerate a model transition. |
| llamacpp-router | Inference runtime | One server switches models in process rather than replacing a pod. | The tested switch was quick, but the tested configuration did not pass GPU-offload settings to child servers, so it is not a serving recommendation based on these results. |
| MIG (partitioned) | Hardware | On supported GPUs, the GPU is divided into hardware-isolated instances with dedicated resources. | Potentially useful where isolation and predictable resource allocation matter; it was not benchmarked in Maher’s setup. |
What time-slicing did to inference performance
The numbers below are measurements from Maher’s 2026 setup, not general performance guarantees. Prefill processes the input prompt; decode generates output tokens. Because prompt processing and generation respond differently to contention, a single throughput figure would conceal an important trade-off.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minute| Test condition | Prefill | Decode | What the result indicates |
|---|---|---|---|
| 27B dense model at Q4, one GB10, no other workload | 768.85 tokens/s | 31.34 tokens/s | Maher’s reported baseline after correcting the configuration. |
| 12B model with an idle 27B co-tenant | 77.9–80.9 tokens/s | 24.1–26.2 tokens/s | Reported as within measurement noise of the 12B model alone: 76.7–77.6 tokens/s prefill and 27.3 tokens/s decode. |
| Four active tenants | 16-fold decline from the comparison used in the report | 7.33 tokens/s, a 4.3-fold decline | A 10,050-token prompt took 209 seconds. |
The idle co-tenant comparison is different from active contention: the 27B model was suspended for the alone measurement, so the 12B model could be compared with and without another resident model that was not doing work. The four-active-tenant result shows the more consequential case when workloads compete. In this test, time-slicing looked more suitable for chat with short prompts than for large-context requests, where prefill slowdown can dominate. That is a workload-specific interpretation, not a promise for other GPUs, runtimes, or prompt patterns.
#1 Best Overall
- Axial-tech fans now feature a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- 2.5-slot design allows for greater build compatibility while maintaining cooling performance
- 0dB technology lets you enjoy light gaming in relative silence
- Dual BIOS switch lets you toggle between Quiet and Performance BIOS profiles
- Dual ball fan bearings last up to twice as long as sleeve bearing designs
How much memory does each GPU-offloaded model copy use?
Maher retracts an earlier claim that duplicate model copies cost about 0.24 times model size on unified memory, compared with 1.16 times on a discrete GPU. A controlled replication on the same node did not reproduce the apparent saving. It used a cached 15.66 GiB GGUF and three instances configured with --n-gpu-layers 99; measured instance costs were 16,727 MiB, 16,433 MiB, and 16,444 MiB. Maher characterizes that as about 1.03 times the model size per instance, with memory use growing linearly.
The report’s later comparison separates GPU-offloaded copies from CPU-resident weights:
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5070 Ti
- Integrated with 16GB GDDR7 256bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
| Hardware and model | Placement | Memory per additional instance, relative to model size | Sharing observed? |
|---|---|---|---|
| GB10, 27B | GPU offload | 1.03× | No |
| GB10, 3B | GPU offload | 1.38× | No |
| RTX 5060 Ti, 3B | GPU offload | 1.16× | No |
| GB10, 3B | CPU-resident weights, --n-gpu-layers 0 |
0.19–0.36× | Yes, in the reported comparison |
The distinction is placement, not simply unified versus discrete memory. The author’s explanation is that mmap page-cache sharing can reduce duplication while weights stay CPU-resident, whereas llama.cpp GPU offload allocates private CUDA device buffers and copies weights there, including on unified memory. In a same-box warm comparison, the GPU-offloaded run reached 99.00 tokens/s decode, compared with 11.36 tokens/s with zero GPU layers. For capacity planning, budget roughly a full model size for each GPU-offloaded copy on either architecture; a larger unified memory pool is not the same as cheaper copies.
How long do model switches take?
Switch latency is part of the effective cost of sharing. A method that conserves residency by unloading one model before loading another may be unsuitable if requests cannot wait through the transition.
Rank #3
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5060
- Integrated with 8GB GDDR7 128bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
| Switch method | Measured transition | Operational behavior reported |
|---|---|---|
| ModelPool, cold swap | 122.0 seconds | Includes pod teardown, scheduling, container startup, and model loading. |
| ModelPool, warm swap | 53.6 seconds | One member is resident at a time; transitions remain substantial even when warm. |
| llamacpp-router, in-process switch | 4.5–4.7 seconds | Much quicker in the test, but the router did not pass --n-gpu-layers to child servers. The resulting inference performance was not a viable GPU-serving result. |
ModelPool’s tested safeguards may matter as much as its timing: drain-before-unload preserved an in-flight 1,200-token generation, simultaneous requests were coalesced into one swap, and a fail-closed path kept the incumbent model resident if its successor could not be reached. These behaviors describe the tested implementation, not every model manager.
Is MIG better than time-slicing?
There is no direct MIG-versus-time-slicing result for Maher’s hardware. The GB10 and RTX systems in the setup reported nvidia.com/mig.capable=false, so MIG was not tested. The measurements therefore cannot show whether MIG would be faster, slower, or otherwise preferable for these LLM workloads.
Rank #4
- Powered by Radeon RX 9070 XT
- WINDFORCE Cooling System
- Hawk Fan
- Server-grade Thermal Conductive Gel
- RGB Lighting
NVIDIA’s Multi-Instance GPU User Guide, updated September 11, 2026, describes MIG on supported GPUs beginning with Ampere as a way to create up to seven GPU instances with isolated paths through cache and memory resources. NVIDIA’s overview describes instances with dedicated compute, memory, and bandwidth. This hardware isolation can support predictable throughput and fault isolation, but instance sizes are fixed or selected rather than dynamically borrowed by a neighboring idle partition. NVIDIA AI Enterprise documentation also describes assigning a hardware-isolated MIG instance to a virtual machine through MIG-backed vGPU.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
NVIDIA’s 2026 technical blog distinguishes software time-slicing and MPS from hardware MIG, and reports about 1.00 request per second per GPU for MIG versus 0.76 for time-slicing in a separate voice-AI pipeline benchmark. That is a vendor result on a different workload and configuration; it is not comparable to the LLMKube measurements and does not supply the missing MIG result for a GB10.
Best Value
- Axial-tech fans now feature a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- Phase-change GPU thermal pad helps ensure optimal heat transfer, lowering GPU temperatures for enhanced performance and reliability
- 2.5-slot design allows for greater build compatibility while maintaining cooling performance
- Dual-ball fan bearings last up to twice as long as standard conventional sleeve bearings designs
- 0dB technology lets you enjoy light gaming in relative silence
Choose by workload, memory, and acceptable interruption
- One model serves demand: use exclusive assignment for the straightforward one-model case.
- Several models need genuine simultaneous access: shared time-slicing can accommodate concurrent demand if the GPU has enough memory, but measure prefill as well as decode—especially for long prompts.
- Models take turns and requests can wait: ModelPool offers one-resident-model turn-taking; decide whether its measured warm transition is acceptable for the application.
- Many models serve short prompts: time-slicing may be a fit in the tested workload, provided the observed active-contention slowdown is acceptable.
- Isolation and predictable resource allocation are priorities: investigate MIG only on hardware that supports it. These tests establish no LLM performance comparison for MIG.
- Considering a router for GPU serving: verify that the runtime passes GPU-offload configuration to the child server; fast switching alone does not establish useful GPU inference performance.
Benchmark carefully before choosing a mode
Several configuration and measurement traps affected the reported tests. They are useful checks when reproducing or evaluating similar systems, but they should be treated as lessons from this setup rather than universal benchmark laws.
- Verify the complete serving specification. Maher’s first 27B baseline measured 12.11 tokens/s decode after an InferenceService spec was cloned without its
speculativeDecodingfield. The report says MTP was a first-class CRD field, not anextraArgsentry. Diff the full spec when cloning benchmark configurations. - Separate cold and warm requests. In the cited GB10 example, cold prefill was 93 tokens/s versus 743 tokens/s warm. A first request after loading a model can misrepresent steady-state behavior.
- Watch for prefix caching. Reusing prompts can trigger prefix caching and skip prompt work, making prefill appear faster than a fresh-prompt test.
- Account for operator reconciliation. In the described setup, manually scaling a deployment with
kubectl scalecould be undone by the operator, which reconciled it to the configured state. - Measure memory with a tool that reports it for the device. The retracted comparison mixed node memory with
nvidia-smi, which reported[N/A]for memory on GB10. A single observation under four-tenant load, no replication, and a comparison between 3B and 27B models compounded the error. - Validate the load harness itself. The author’s synthetic load test missed defects in an agent-and-reviewer harness, including a reviewer receiving the base branch rather than the actual diff. A plausible throughput result does not prove the test exercised the intended code or configuration.
Maher reports throughput and memory measurements, not output-accuracy testing. These results therefore do not establish that sharing preserves output accuracy.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →
