Recommended Free Tools
A system can detect eight GPUs while an AI job uses only one, alternates between devices, or spends most of its time waiting. Detection is only the first condition. The application must launch work on each device, divide the input correctly, move data efficiently between GPUs, and keep every rank supplied with CPU, memory, and (for multi-node jobs) network resources.
Troubleshoot in that order: verify the execution model and process-to-device mapping, measure a complete iteration with a timeline, inspect GPU topology and peer-to-peer paths, then investigate NUMA and network behavior. Change tuning settings only after the trace identifies a bottleneck.
As an Amazon Associate I earn from qualifying purchases.
What “using every GPU” actually requires
Four separate conditions determine whether a multi-GPU workload makes progress on all devices:
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →- Placement: the framework must create the intended number of workers and assign each one to the correct GPU.
- Partitioning: workers must receive distinct input or requests when the execution model requires it.
- Communication: gradients, activations, parameters, or requests must travel over an effective GPU-to-GPU or GPU-to-network path.
- Supply: CPU launch threads, data loading, host memory, and network transfers must keep the GPUs fed.
A utilization percentage is an observation of one part of that chain, not a diagnosis. A GPU can report low utilization because the CPU is launching kernels slowly, because input batches arrive late, because synchronization is waiting for another rank, or because the workload is simply too small for the device.
#1 Best Overall
- Axial-tech fans now feature a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- 2.5-slot design allows for greater build compatibility while maintaining cooling performance
- 0dB technology lets you enjoy light gaming in relative silence
- Dual BIOS switch lets you toggle between Quiet and Performance BIOS profiles
- Dual ball fan bearings last up to twice as long as sleeve bearing designs
Step 1: confirm the execution model before changing hardware
Start by identifying whether the job is single-process, multi-process, data-parallel, model-parallel, pipeline-parallel, or an inference service with its own scheduler. The same machine-level symptom has different fixes in each model. Do not assume that a library uses PyTorch DistributedDataParallel (DDP), or that an inference server follows DDP rules.
For PyTorch DDP, verify ranks and devices
In DDP, normally one process owns one GPU. The process should select its device before initializing the distributed process group. The official PyTorch tutorial warns that failing to do so can lead to hangs or excessive memory use on GPU 0. Check all of the following in the startup log for every process:
- the number of processes launched on the node;
- the process rank and world size;
- the local rank and the GPU selected for that process;
- the visible-device list after any
CUDA_VISIBLE_DEVICESsetting; - the device on which the model and its input tensors are created.
The process count is not proof that useful work is distributed. A four-process job can still place all model data on one device, or all ranks can read the same samples. For map-style datasets, use a distributed sampler or the framework’s equivalent so each rank receives its intended portion of the input. Confirm that the sampler is configured for the current world size and that its epoch is advanced as required by the training loop.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Respect exclusive GPU ownership in multi-process NCCL jobs
PyTorch’s distributed documentation recommends NCCL for GPU distributed-training performance. It also cautions that, when multiple processes use NCCL on one machine, each process must have exclusive access to every GPU it uses. Sharing a GPU between processes can cause invalid usage or a deadlock. This guidance applies to the stated NCCL and multi-process context; an inference framework with a different scheduler may have different rules.
Check inference-specific scheduling
Inference systems may use one worker per GPU, a request router, tensor parallelism, or a batching scheduler. Inspect that product’s worker count, device list, and parallelism mode rather than applying DDP assumptions. A router can show all GPUs as available while sending requests to only one because the request batch is too small, a worker is unhealthy, or a concurrency limit is set below the number of devices.
A minimal launch audit
- Record the exact launch command, framework version, backend, and environment variables that affect device visibility.
- At process startup, print rank, world size, local rank, selected device, model device, and the first input tensor’s device.
- During one iteration, log the number of samples or requests handled by each rank.
- Verify that a failed or idle rank is not silently restarting, waiting at a barrier, or exiting after an initialization error.
If this audit shows one process or one device, fix the launch and placement configuration first. Fabric tuning cannot make an unassigned GPU perform work.
Step 2: observe a complete iteration, not one utilization sample
Collect a representative interval that includes data loading, forward work, backward work, synchronization, and the optimizer step. Record per-GPU utilization and memory alongside step time, CPU usage, data-loader timing, and (for multi-node jobs) network activity. A single snapshot can catch a crashed worker but cannot explain intermittent idle periods.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Rank #2
- 6 HDMI Multi-Monitor Mastery: Unleash unparalleled productivity with six HDMI ports on a single RX 580 graphics card. Simultaneously drive up to four independent displays, creating an immersive multi-screen workstation for trading, programming, digital signage, or heavy multitasking—no adapters or extra cards required.
- Powerful RX 580 Graphics with 8GB GDDR5 Memory: Equipped with the RX 580 GPU and 8GB of GDDR5 video memory, this card delivers smooth, responsive performance across all connected displays. Handle 4K multimedia, complex spreadsheets, design software, and data-intensive applications without lag or stuttering.
- Instant Plug-and-Play Productivity: True plug-and-play simplicity lets you set up a sophisticated multi-monitor array right out of the box. Effortlessly expand your screen real estate and workflow efficiency with zero driver hassle—the most cost-effective way to boost productivity instantly.
- Standard-Profile Design with Active Cooling: Built on a reliable standard-profile form factor, this card ensures broad compatibility with most standard desktop PC cases (not suitable for SFF cases). Active cooling technology maintains optimal temperatures for stable, long-duration performance.
- Optimized Power Efficiency for Easy Upgrades: Engineered for low power consumption, this RX 580 variant draws all necessary power directly from the PCIe slot—no external power connectors needed. A safe, energy-efficient, and simple upgrade path for nearly any standard desktop system.
Useful first measurements
| Question | Measurement | What it can and cannot tell you |
|---|---|---|
| Are devices active at all? | nvidia-smi dmon -s u |
Shows a time series of GPU activity; it does not identify whether idle time comes from input, launches, synchronization, or communication. |
| What are all GPUs doing? | nsys profile --gpu-metrics-devices=all |
Collects GPU metrics for a timeline investigation; choose a bounded representative run to keep the trace manageable. |
| Where are gaps? | Nsight Systems timeline | Shows CPU launch gaps, GPU kernel intervals, synchronization, and idle regions; it is the evidence needed before choosing a fix. |
| Is memory placement plausible? | Per-device memory usage from your framework and system monitor | Uneven memory can indicate incorrect placement or a model-parallel design, but equal memory does not prove equal computation. |
These commands are examples from NVIDIA’s CUDA Graph performance guidance. They are diagnostic starting points, not a universal utilization target. Capture enough iterations to include warm-up effects, steady-state batches, and any periodic evaluation or checkpointing.
Read the timeline by its idle pattern
- Long CPU gaps before kernels: investigate Python or host-side work, data loading, synchronization in the input pipeline, and excessive kernel-launch overhead.
- GPU kernels run continuously but steps remain slow: the workload may be compute-bound, memory-bandwidth-bound, or performing expensive communication; higher displayed utilization alone will not establish good scaling.
- One rank is active while others wait: look for imbalanced batches, straggling data loaders, unequal work, or a synchronization point.
- All ranks pause at similar intervals: inspect collective communication, input stalls, checkpointing, and host or network contention.
- Short bursts separated by gaps: determine whether the CPU launch path or a small batch is limiting occupancy.
CUDA Graphs can reduce repeated CPU launch overhead, but only when launch overhead is the limiting region and the captured graph covers the work that is actually idle. If kernels already occupy the GPU, or the graph excludes the bottleneck, graph capture is unlikely to solve the observed problem.
Step 3: verify single-node GPU communication and topology
Distributed training often exchanges gradients or parameters between GPUs. On one host, that traffic may use NVLink or PCIe peer-to-peer (P2P) access, depending on the physical topology, driver support, and CUDA’s reported capabilities.
Check the peer matrix
For NCCL workloads, inspect the relevant peer paths with:
nvidia-smi topo -p2p pfor PCIe peer-to-peer capability;nvidia-smi topo -p2p nfor NVLink peer-to-peer capability.
NCCL generally favors P2P when CUDA reports direct access between a pair. An OK result is useful evidence that the pair can use that access path, but it does not validate the entire system. A job can still be slow because of topology, a congested path, synchronization imbalance, CPU affinity, or an unrelated input bottleneck.
Interpret topology rather than treating it as a ranking
GPUs connected through the same NVLink fabric may communicate differently from GPUs that must cross a PCIe switch or CPU root complex. The relevant comparison is the path used by the ranks in your job, not the number of GPUs visible in the machine. Record the GPU-to-GPU matrix, PCIe layout, and which process owns each device before changing placement.
Investigate IOMMU and PCIe ACS only in the right environment
NCCL’s GPU troubleshooting guidance identifies bare-metal IOMMU settings and PCIe Access Control Services (ACS) as possible causes of broken or unexpectedly slow peer-to-peer traffic. On a physical Linux host, inspect the kernel command line and kernel messages for IOMMU state, and inspect PCI bridge configuration with lspci. The exact output depends on the distribution, kernel, motherboard, and accelerator.
Rank #3
- 6 HDMI MULTI-MONITOR GRAPHICS CARD: Features 6 native HDMI outputs and supports up to six monitors simultaneously, making it ideal for multi-screen office, trading, monitoring and digital signage setups.
- RADEON R7 350 WITH 2GB GDDR5: Powered by the Radeon R7 350 GPU with 2GB GDDR5 memory, providing reliable display performance for everyday productivity, multi-window applications and multi-monitor workstations.
- SUPPORTS UP TO 6 DISPLAYS: Connect up to six HDMI monitors directly without additional HDMI adapters. Supports up to 1920 × 1080 at 60Hz per display on compatible systems and monitors.
- PCIe x16 SLOT POWERED: Installs into a compatible PCI Express x16 slot and does not require an additional external power connector. Maximum graphics card power consumption is approximately 60W.
- BUILT FOR MULTI-SCREEN APPLICATIONS: A practical solution for office workstations, stock trading setups, monitoring systems, digital signage, presentation displays and other applications that require multiple independent screens.
Do not blindly disable IOMMU or ACS. Virtual machines have different constraints: NCCL documents that ACS is required in some virtualized configurations, so advice appropriate to bare metal can damage a VM’s device isolation or prevent access. Establish whether the job runs on bare metal, in a VM, or in a container, and understand the operational and security consequences before altering firmware or kernel settings.
Check topology visibility in containers and VMs
NCCL uses /sys to discover PCI topology. Ensure that /sys is correctly mounted and exposes the expected devices inside a container or virtual machine. A virtualized or incomplete PCI view can make NCCL select a suboptimal path even when the host’s physical topology is favorable.
GPU-to-NIC GPUDirect RDMA is a separate path from local GPU P2P. It has additional prerequisites, including compatible NIC and driver support and either the nvidia-peermem module or a supported DMA-BUF configuration. A healthy local peer matrix does not prove that GPUDirect RDMA is available.
Step 4: separate computation from communication in multi-node jobs
When ranks span hosts, a slow step can reflect GPU computation, interconnect bandwidth, network latency, or host placement. NCCL’s troubleshooting documentation separates GPU, networking, runtime/MPI, diagnostics, and performance-tuning paths. Follow the path that matches the symptom instead of changing collective or transport variables at random.
Check network fabric with context
For InfiniBand or another supported RDMA fabric, the NCCL performance guidance lists ib_write_bw and ib_write_lat as ways to measure fabric bandwidth and latency. Run such tests only with the fabric, drivers, permissions, and topology they expect; a result from a different interface or an improperly selected port is not evidence about the path used by the AI job.
Also verify which network interface NCCL selects, whether the interface is up on every host, and whether firewalls, containers, or namespace rules block the required traffic. Capture the NCCL logs with a bounded test so you can correlate rank, host, interface, and collective timing.
Keep each rank close to its GPU and NIC
NCCL’s tuning guidance notes that CPU cores and host memory for each rank should generally be close to its GPU and, in multi-node jobs, to its NIC. Poor NUMA placement can add host-memory latency and consume inter-socket links even when the GPUs and network are fast.
Rank #4
- Why RX 560] The RX 560 is an entry-level discrete graphics card with a 14nm process and Polaris 12 core. If you need to increase the resolution in your office, if you are not a game enthusiast and just need to play games like LOL in your spare time, rx 560 is the best choice for you.
- [Support 4 ports] The RX 560 2gb includes 4 HDMI ports. If speculating on stocks is part of your job and you need 4 screens to watch the stock market at the same time and need a stable display, the Rx 560 can fulfill your needs and provide an excellent experience.
- Low Power Consumption] The RX560 does not require an external power connector, and uses only the power supplied by the PCI Express X16 slot, which is rated at a maximum of 50W. All-solid-state capacitors ensure low power consumption and high performance.
- Faster Heat Dissipation】This RX 560 graphics card comes with a cooling fan that provides more airflow for a cool and quiet gaming experience. High-quality electrical components and PCB board ensure its safety and stability.
- Supported Operating Systems: This 4-display video card is available for the following operating systems: Windows 11-64 bit, Windows 10 64 bit, Windows 7 64 bit.（SIZE:180*120*40mm(full height baffle size)）
Map each GPU, NIC, CPU socket, and NUMA node. Then compare that map with the CPU affinity and memory policy used by the launcher or scheduler. Correct locality only after confirming it is the bottleneck; an affinity change can help one topology and hurt another.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.When the workload is simply too small
A powerful GPU can remain lightly occupied when batches, models, or inference requests are small. This is not automatically a communication failure. If each device receives only a small amount of work, distributing it can add synchronization and transfer overhead without creating enough parallel computation to offset the cost.
Free tools Windows power users keep installed
One-click scans. No signup required.
NVIDIA describes Multi-Instance GPU (MIG) as a way to run multiple small workloads concurrently on separate GPU instances when a full GPU is underused. MIG mode does not provide GPU-to-GPU P2P and does not support multi-GPU or multi-node training. Therefore, MIG is an option for consolidating independent small workloads, not a fix for a distributed training job that fails to engage all GPUs.
For a small workload, compare three measured alternatives: run it on one GPU, increase batch or request concurrency if memory and latency requirements allow, or place independent jobs on separate instances. Keep the objective explicit—throughput, latency, isolation, or utilization—because the best choice differs for each.
Tune only after the bottleneck is established
Use a measured baseline and make one change at a time. Save the launch command, software versions, topology output, step-time distribution, and per-GPU observations before each experiment.
- Fix placement: correct process count, rank/device mapping, sampler behavior, and exclusive ownership.
- Fix the supply path: address data-loader stalls, CPU affinity, host-memory placement, or launch gaps shown by the timeline.
- Fix communication: investigate P2P, topology, IOMMU/ACS context, NIC selection, and fabric performance.
- Retest at steady state: compare the same batches, precision, sequence lengths, and number of steps.
- Only then tune: NCCL environment variables and transport choices are version- and topology-dependent; arbitrary settings can make performance worse.
A successful change should improve a metric tied to the diagnosed bottleneck—such as shorter step time, fewer idle gaps, lower collective latency, or better scaling—not merely raise a dashboard percentage.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Symptom-to-check decision table
| Observed symptom | First checks | Do not conclude yet |
|---|---|---|
| Only one GPU has memory and activity | Process count, rank/device logs, visible devices, model and input tensor devices | That the driver or PCIe fabric is defective |
| All GPUs show memory use but some are mostly idle | Per-rank sample counts, sampler configuration, timeline, synchronization waits | That low utilization means the GPUs are unusable |
| GPU kernels are separated by long gaps | CPU launch path, data loading, host preprocessing, and Nsight Systems trace | That CUDA Graphs will help without proving launch overhead is the bottleneck |
| Single-node collectives are unexpectedly slow | P2P matrix, NVLink/PCIe topology, bare-metal versus VM/container context, IOMMU/ACS evidence | That an OK peer result validates every system path |
| Multi-node steps wait at collectives | NCCL network logs, interface selection, fabric bandwidth/latency, rank-to-NUMA/NIC locality | That changing an environment variable is a universal remedy |
| Every GPU is lightly loaded on a small job | Batch/request size, concurrency, single-GPU baseline, memory and latency limits | That adding GPUs must increase throughput |
What to collect before asking for help
A reproducible report is more useful than a screenshot of a utilization graph. Include:
- GPU model, count, driver and CUDA versions, and whether the host is bare metal, virtualized, or containerized;
- framework and version, backend, training or inference mode, and the exact launch command;
- world size, rank-to-device mapping, visible-device settings, and sampler or batching configuration;
- per-GPU utilization and memory over a representative interval, plus step-time or request-latency measurements;
- an Nsight Systems timeline or equivalent trace showing CPU, GPU, synchronization, and communication regions;
nvidia-smi topo -p2p pand, where relevant,nvidia-smi topo -p2p noutput;- for multi-node runs, host-to-rank mapping, selected NICs, NCCL diagnostics, fabric test context, and NUMA affinity;
- errors, hangs, repeated worker restarts, and the exact point in the iteration where progress stops.
With those details, you can distinguish an unassigned GPU from an undersized workload, a CPU/input bottleneck, a topology problem, and a network or NUMA problem. Without them, a higher utilization number or a new transport setting is only a guess.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
