Free tools Windows power users keep installed
One-click scans. No signup required.
The reliable way to decide between more memory and pooled memory is to run a controlled load test with the exact model, quantization, prompt lengths, concurrency and service-level objectives you intend to serve. Record GPU memory, model-weight size, KV-cache use, preemptions, queue depth, time to first token (TTFT), p99 latency, throughput and active requests as time series. Persistent vllm:kv_cache_usage_perc above 0.9, any nonzero vllm:num_preemptions, or a growing waiting queue while latency misses its objective indicates a memory-capacity limit. Repeat the same test with the proposed larger GPU, additional GPU, memory fraction, swap tier or memory-pooling fabric, then choose the least complex design that meets latency, throughput and burst-headroom targets.
What “running out of memory” means on an AI server
AI inference memory pressure is a workload condition, not a single number shown by a monitoring tool. A server must hold model weights, temporary execution data, the key-value (KV) cache for active sequences, framework overhead and, in some designs, data copied between GPU and host memory. The required amount changes with model revision, quantization, context length, batch policy and concurrency.
As an Amazon Associate I earn from qualifying purchases.
Model weights create the initial fit requirement. After weights are loaded, the serving engine allocates space for the KV cache and other runtime buffers. Longer contexts and more simultaneous requests consume more KV entries. A model can therefore load successfully at low traffic and still run out of usable memory during a burst. Conversely, a GPU can show high utilization while retaining enough memory; that case may be compute-bound rather than a reason to buy more memory.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Host RAM matters when the runtime, operating system, staging buffers or a swap tier use it. It is not equivalent to high-bandwidth GPU memory. Any design that moves data between GPU and CPU must be evaluated for transfer overhead and tail latency.
#1 Best Overall
- Valued Carpenter Pencil Set: You will get 2 pcs solid carpenter pencils with 26 piece 2.8 mm refills, 1 replaceable sharpener, 1 plastic storage box.The complete carpenter pencils combination allows you to finish your work faster and more easily
- Deep Hole Marker Pencil: The deep-hole construction pencils adopts 45mm elongated tip design, which is more convenient to mark in the small hole or in other tight areas that other carpenter markers cannot reach
- Carpenter Pencils with Sharpener: The sharpener is screwed into the top of the work pencil, which won't get lost either. Built-in pencil sharpener that keep the lead with pointed and smooth to Improves line of sight in fine work
- Stronger Solid Lead: This work pencil is matched with a 2.8 mm thick lead , which is much thicker and stronger during the drawing process of construction work, it will not break or damage easily
- Marks on Various Surfaces: 3 colors solid construction pencil can marks on various surfaces,such as metal, plastic, wood, paper etc. Ideals for woodworkers, contractors, craftsmen, builders, merchants and masons
Metrics that demonstrate a capacity limit
Collect the following metrics continuously during a controlled test. A single peak reading can miss the queueing and preemption behavior that appears only after several requests overlap.
| Metric | What it tells you | Capacity signal |
|---|---|---|
| GPU memory used/free | Physical headroom on each device over time | Headroom approaches zero at the same load where service objectives fail |
| Model-weight footprint | Memory consumed before request-dependent cache growth | Weights plus required runtime buffers cannot fit on one device |
vllm:kv_cache_usage_perc |
Fraction of the allocated KV cache in use | NVIDIA AIPerf documents a value above 0.9 as KV cache near full capacity |
vllm:num_preemptions |
Requests or sequences forced out because memory is unavailable | NVIDIA AIPerf identifies any value above zero as memory pressure causing preemptions |
| Waiting requests and queue depth | Demand that cannot start immediately | A growing queue indicates saturation; correlate it with memory and latency |
| TTFT | Delay before the first generated token | Rises when requests wait for memory, scheduling or transfer resources |
| p99 latency | Tail response time experienced by the slowest one percent | Breaches the service objective when bursts exhaust headroom |
| Token throughput | Useful output rate under the selected load | Stops increasing or falls as preemptions and queueing begin |
| Active requests and concurrency | How many sequences hold cache space simultaneously | Shows whether the failure is tied to parallel demand |
| GPU utilization | Compute activity, distinct from memory capacity | High utilization with spare memory points to compute or scheduling limits instead |
Treat the threshold as persistent behavior, not a one-sample alert. A brief excursion above 0.9 during a controlled ramp is less conclusive than a sustained value accompanied by preemptions, queue growth and a p99 violation.
How to run a measurement that answers the hardware question
1. Freeze the test matrix
Write down the model revision, tokenizer, quantization, serving-engine version, prompt distribution, input and output token limits, target concurrency, batch policy and service-level objectives. Keep these fixed while comparing designs. Changing context limits or quantization between runs can make a memory improvement appear larger or smaller than it really is.
2. Establish a low-load baseline
Start below the expected operating point with the same model and request shape. Record GPU memory used and free, model-weight footprint, KV-cache percentage, host-memory use, TTFT, p99 latency, throughput, queue depth and active requests. This baseline separates the memory occupied by the model from memory that grows with traffic.
3. Increase concurrency and context length in steps
Raise one pressure variable at a time when possible. For example, hold context length constant while increasing concurrent requests, then repeat with longer contexts. At each step, allow the system to reach a steady state and capture time-series data rather than a single reading. Include a burst that represents the largest arrival rate your service-level objective permits.
4. Mark the first capacity breach
Record the earliest load at which one or more of these conditions persist: vllm:kv_cache_usage_perc stays above 0.9, vllm:num_preemptions becomes nonzero, waiting requests grow, or p99 latency exceeds its objective. Note the corresponding memory, concurrency, context length, throughput and queue values. This point is your measured capacity boundary for the current configuration.
Rank #2
- Ergonomically Designed: Work in tight areas with a compact design that gets into tough spots
- Both tools are designed to fit into difficult to reach spaces
- Complete a wide range of applications with a high-performance motor that delivers up to 300 UWO, and an impact driver delivering up to 1,400 in-lbs of torque
- Both the DEWALT impact driver and drill driver feature integrated LED work lights with a convenient 20-second delay, ensuring enhanced visibility in dimly lit or challenging work areas
- One-Handed Loading - Keep one hand free with a 1/4 in. hex chuck that accepts 1 in. bit tips
5. Distinguish memory pressure from compute saturation
Compare memory readings with GPU utilization, token throughput and queueing. High compute utilization with KV-cache headroom, no preemptions and a stable queue indicates a compute or scheduling bottleneck. Adding memory will not fix that bottleneck. Conversely, a full cache, preemptions and a rising queue while compute utilization is moderate is evidence for additional capacity or a design that can share it.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →6. Test the proposed alternative under the identical load shape
Repeat the matrix with the larger-memory GPU, an added GPU, a different gpu_memory_utilization fraction, a memory-swap tier, NVLink/NVSwitch or a CXL configuration. Use the same prompts, token limits, concurrency steps and burst. Compare the first breach point, not just idle memory, and record any changes in p99 latency, TTFT, throughput, queue depth and failure behavior.
7. Select the least complex design that meets the objectives
Choose the option that satisfies required concurrency, p99 latency, throughput and burst headroom while preserving a documented failure mode. Include power, cooling, topology, firmware, driver, operating-system, scheduler and serving-engine compatibility in the decision. A design that works only in a narrow steady-state test is not adequate capacity.
How vLLM allocation choices affect the result
vLLM allocates model weights before sizing the KV cache. The gpu_memory_utilization setting controls the usable fraction after profiling; it does not create additional physical memory. Raising the fraction can leave less safety margin for framework overhead and other processes, while lowering it deliberately reserves headroom at the cost of cache capacity.
The max-model-len setting determines the maximum context length the engine must be prepared to support. A larger value reserves more KV-cache memory even if typical requests are shorter. When a server appears memory-bound, compare the configured maximum with the actual prompt distribution. Reducing an unjustifiably high maximum can recover capacity, but it changes the service contract and must be validated with representative long requests.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteMeasure these settings together. A larger GPU-memory fraction may postpone preemptions in a test, yet leave too little room for monitoring agents or another workload in production. A smaller maximum context may improve headroom but cause over-limit requests to be rejected. Document both the setting and its operational consequence.
Rank #3
- 【Great Compatibility】This Katerk 1/4 inch hex shank bit holder is specifically designed for 1/4 inch hex shank drill bits. It's compatible with most 1/4 fast hex handles, hex sockets, various electric screwdrivers, and handheld screwdrivers. The bit holder makes it a valuable addition for any handyman.
- 【Secure and Safe】Built with a secure backup nut design, each drill bit holder securely locks onto your bits, ensuring they stay firmly in place. Additionally, our bit holder incorporates a high-quality steel ball rolling design that holds up to several kilograms of weight, ensuring your various drill bits don't fall off.
- 【Easy One-Handed Operation】The bit holder for impact driver allows you to change bits single-handedly, simplifying your workflow. Its multi-color design further allows for quick identification of the drill bit you need.
- 【Compact and Convenient】Thanks to its compact size, this 1/4 inch bit holder is easy to carry around. The bit holder allows for easy attachment to various tools, making this a convenient addition to your construction accessories. The Katerk bit holder is cast from high-quality alloy material, promising a long product lifespan. Despite its rugged strength, the bit holder remains lightweight, making it portable.
- 【Cool Christmas Gift For Men Stocking Stuffers】 This screwdriver bit holder, driver bit holder, impact bit holder, can be given as a gift to your loved one, especially for anyone involved in construction or electrical work. It's a must-have for stocking stuffers for men and women, tools gifts for dad, tech gadgets for men, gifts for dad, gifts for him, gifts for husband, gifts for boyfriend, cool gadgets for men, and cool gifts for dad.
When adding physical GPU or system memory is the right answer
Use a larger-memory accelerator for a hard fit limit
Physical expansion is the clearest choice when one model replica cannot load, KV-cache allocation fails, or measured pressure remains sustained on a single device after sensible context and batching limits are set. A larger-memory accelerator removes a hard per-device limit and can avoid the latency cost of remote or host-memory access.
| GPU reference | Memory specification | Qualification |
|---|---|---|
| H100 | 80 GB HBM3 per GPU | NVIDIA HGX enterprise reference architecture |
| H200 | 141 GB HBM3e per GPU | NVIDIA HGX enterprise reference architecture |
| B200 | 180 GB HBM3e per GPU | NVIDIA HGX enterprise reference architecture |
These figures are per-GPU reference capacities, not a guarantee that an existing server can accept the card. Verify chassis space, power delivery, cooling, bus and firmware support, driver and serving-engine compatibility, and the required operating-system and scheduler integration. A card with more memory can still underperform if the host cannot sustain its power or if the interconnect is inadequate.
Add GPUs when parallelism is already supported
An additional GPU can increase aggregate capacity when the serving stack supports tensor, pipeline or other model-parallel placement. It is not automatically equivalent to one larger memory pool: weights and cache may be partitioned, replicated or communicated between devices. Measure the resulting inter-GPU traffic, synchronization, p99 latency and throughput under the same load.
Expand host memory for host-side requirements
More system RAM is appropriate when profiling shows the host is the limiting tier—for example, runtime staging, CPU-side batching or a deliberate swap design. It does not replace HBM for a model path that requires GPU-resident weights and cache. Validate paging and transfer behavior at peak concurrency rather than relying on total installed RAM.
When pooling GPU memory makes sense
NVLink and NVSwitch
NVIDIA describes NVSwitch as a high-bandwidth, low-latency fabric for peer-to-peer GPU communication and pooled GPU memory on supported systems. This is useful when model parallelism or shared state needs faster access than a conventional host path can provide. The benefit depends on the exact GPU platform, switch topology, link configuration and software support.
NVIDIA’s IMEX guidance describes a multi-node NVLink design in which GPUs can access memory from other nodes within an NVLink domain. Such a design extends the multi-GPU peer-to-peer programming model across compute nodes, but it is not a universal feature of all GPUs or servers. Check the supported hypervisor, operating system, fabric services and scheduler before treating remote memory as available capacity.
Rank #4
- Long Nib and Deep Hole Marker: Our mechanical carpenter pencil with 45mm nib is designed for easy marking of deep holes or narrow areas. These construction pencils are the great choice for woodworking tools, construction tools, carpenter tools, contractor tools, wood carpentry tools and architect tools
- Extra Refills in 2 Colors for Versatile Marking: The construction mechanical pencil comes with 12 extra 2.8mm refills, including 6 red and 6 black refills. The black refill is suitable for light surfaces, while the red wax is perfect for dark surfaces. Our carpenter mechanical pencil makes sure that you'll have an ample supply for extended use
- Built-in Sharpener: Our construction pencil comes with a built-in sharpener to ensure the mechanical pencil tip is always sharp and ready for use. Never buy an extra pencil sharpener again. A great tool for any woodworker pencil, contractor pencils. The refill can easily be extended or retracted with a simple click of the pencils mechanical, allowing you to work more efficiently and accurately
- Portable Clip Design: Our deep hole construction pencil features a portable clip design, easy to carry and attach to your pocket or tool box, so that you can keep the carpenter pencils mechanical close at hand, making it a convenient tool to have on the go. Great gifts choice for carpenters
- Stronger Pencil Lead: The black refills are made of lead, sturdy and smooth. The red refills are made of wax, clear and light. These marking pencils are much thicker and stronger than normal pencils during the marking process of construction work, suitable for various surfaces, such as glasses, metal, boards, floors, walls, furniture, etc. The written marks can be easily wiped with a wet paper towel when needed
CXL host-memory expansion and pooling
Compute Express Link (CXL) is an open interconnect standard described as supporting accelerator attachment, host-memory capacity expansion, multi-server resource pooling and peer-to-peer communication. It can add a shared memory tier or increase effective host capacity. CXL should not be assumed to provide GPU-HBM latency or bandwidth. Benchmark the actual model path, including paging, cache movement and tail latency, with the same workload used for local-memory testing.
Pooling is a topology and software decision
Pooling succeeds only when hardware links, firmware, drivers, operating system, hypervisor, fabric services and serving software agree on the memory model. Failure isolation also changes: a switch, fabric service or remote node may become a dependency for every replica using the pool. Document what happens when a link, pooled device or service disappears, and test recovery rather than measuring only the healthy path.
Fractioning and swap: useful stopgaps with measurable costs
NVIDIA Run:ai documents GPU-memory fractions for dividing one GPU among heterogeneous workloads. Fractioning can improve utilization and shorten startup waits when workloads have different footprints, but each fraction reduces the peak memory available to an individual workload. Test isolation, admission behavior and burst handling when several fractions become busy simultaneously.
NVIDIA Run:ai also documents GPU memory swap, which extends effective GPU memory into CPU memory so more workloads can run on the same physical GPU. Swap can postpone a hardware purchase, but every evicted or transferred block introduces movement across the GPU–CPU path. Measure transfer overhead, TTFT, p99 latency, throughput and fairness under peak load. If swap prevents an out-of-memory error but causes unacceptable tail latency, it has not met the service objective.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Expansion versus pooling: a decision comparison
| Decision axis | Physical expansion | Pooling or sharing |
|---|---|---|
| Model-fit capacity | Raises capacity on one device or adds explicitly partitioned devices | Combines capacity only when the software can address remote memory |
| KV-cache headroom | Usually predictable from local capacity | Depends on placement, traffic and remote-tier policy |
| p99 latency | Often easier to bound with local memory | Can rise with fabric contention or paging |
| Throughput | Improves when the added device removes the measured bottleneck | Improves only if communication overhead is lower than the capacity gain |
| Memory bandwidth | Uses the accelerator’s local HBM bandwidth | Uses an interconnect or host tier for remote accesses |
| Topology dependency | Requires compatible chassis, bus, power and cooling | Also requires supported NVLink/NVSwitch or CXL topology and fabric services |
| Software support | Requires drivers and serving-engine support for the GPU | Requires memory-aware placement, pooling APIs and supported platform software |
| Failure isolation | A failed device generally affects its local replica | A failed fabric, switch or pooled node can affect multiple consumers |
| Operational complexity | Hardware replacement and capacity planning | Topology, firmware, scheduler and fabric lifecycle management |
| Cost per served request | Determine from purchase, power, cooling and achieved throughput | Determine from fabric, memory devices, software and measured throughput |
Use the table as a measurement plan rather than a universal ranking. The cheaper design on a bill of materials can cost more per request if remote accesses cut throughput or force a higher p99 latency target.
Diagnose common symptoms
The model will not load
Confirm the model-weight footprint and available memory before considering cache. If weights plus runtime overhead exceed one device, reduce the model footprint through an approved quantization or placement strategy, use a larger-memory accelerator, or deploy supported model parallelism. Increasing concurrency cannot solve a static fit failure.
Best Value
- Milwaukee Ink all Fine Point Marker, Black, 4 Per Pack
- 4 per pack Features Clog Resistant Marker Tip Writes through Dusty, Wet and Oily Surfaces Durable Marker Tip for Writing on Concrete, OSB and Rough Surfaces
- Clog resistant tip writes on dusty, wet and oily surfaces and is optimized for rough surfaces such as OSB, cinderblock and concrete
- Hard hat clip- attaches for easy access
- Quick dry time with reduced smearing and marking
KV-cache allocation fails after the model loads
Check max-model-len, requested concurrency and the configured gpu_memory_utilization. A maximum context that is much larger than real traffic can reserve unnecessary cache. If the required production context cannot fit even after justified tuning, add local capacity or validate a supported pooling design.
Preemptions appear and the queue grows
Any nonzero vllm:num_preemptions is a memory-pressure signal in NVIDIA AIPerf’s metric reference. Correlate the first preemption with KV-cache percentage, active requests and GPU memory. If the pattern persists at the target load, add capacity, reduce simultaneous cache demand, or use a tested memory tier; do not treat a higher queue limit as a capacity fix.
GPU utilization is high but memory is available
Compare token throughput and latency with the memory metrics. High compute utilization, stable cache usage, no preemptions and a bounded queue indicate a compute or scheduling limit. Investigate batching, kernel efficiency, request scheduling or additional compute before buying memory.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallHost memory is full during pooling or swap
Determine whether host use comes from the operating system, staging buffers, CXL capacity or swap traffic. Check paging rate and transfer latency during the burst portion of the test. Add host memory only when it removes the measured host-side limit without violating p99 latency.
A practical decision worksheet
- Write the exact model revision, quantization, context limits, concurrency target and p99 objective.
- Measure local GPU memory, model weights, KV-cache percentage, preemptions, queues, TTFT, p99 latency, throughput and active requests at low load.
- Ramp concurrency and context length separately, then test the expected burst.
- Mark the first sustained cache value above 0.9, any preemption, queue growth or latency breach.
- Check whether compute is saturated while memory remains available.
- For expansion, verify GPU model, chassis, power, cooling, bus, firmware, driver and operating-system support.
- For pooling, verify NVLink/NVSwitch or CXL topology, fabric services, hypervisor, scheduler and serving-engine support.
- Repeat the identical test with the candidate design and compare headroom, p99 latency, throughput, queueing and failure behavior.
- Record the capacity boundary, recovery procedure and the load level at which admission control should reject new work.
How to interpret the final result
Choose physical expansion when the model or required KV cache cannot fit locally, or when local memory pressure is the demonstrated cause of latency and throughput failure. Choose pooling when the workload can exploit the supported fabric, the measured bandwidth and latency meet the service objective, and the operational team accepts the added topology and failure dependencies. Choose fractioning or swap only when their measured transfer and isolation costs remain within the same objectives.
The decision is complete only after the candidate platform survives the same representative prompts, context lengths, concurrency ramp and burst test as the original server. Capacity is the point at which the service still meets its objectives with documented headroom—not the largest memory number printed on a specification sheet.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →
