For most local large-language-model work on a Ryzen AI Max+ PC, start with the integrated-GPU route: install a ROCm release that lists your exact Ryzen AI Max configuration, then use a ROCm-enabled llama.cpp build. The NPU is a separate path, using ONNX Runtime GenAI and only the Ryzen AI processors, operating systems and models listed in AMD’s compatibility tables. Check those live matrices before installing anything; the correct packages depend on your precise APU and OS.
Decide which processor should run the model
Ryzen AI Max+ systems expose more than one accelerator. Treating the GPU and NPU as interchangeable leads to failed installs and misleading expectations. AMD’s LLM deployment documentation separates the execution modes as follows:
As an Amazon Associate I earn from qualifying purchases.
| Route | Runtime and framework | Compute device | When it fits |
|---|---|---|---|
| GPU LLM inference | llama.cpp with ROCm |
Integrated GPU | The primary route to investigate for a local LLM on Ryzen AI Max+; exact OS and system support must be confirmed in AMD’s matrix. |
| NPU-only | ONNX Runtime GenAI | NPU | Supported Ryzen AI 300 STX/KRK configurations and models. AMD’s Linux examples currently use this mode. |
| Hybrid NPU plus GPU | ONNX Runtime GenAI | NPU and integrated GPU | Use only where AMD lists the processor, model and operating-system combination as supported. |
| PyTorch development | ROCm PyTorch | Integrated GPU | AMD lists Ryzen AI Max 300 series APUs for PyTorch on Linux and Windows in ROCm 7.2.1. |
| CPU fallback | OGA or llama.cpp |
CPU | A compatibility baseline when an accelerator path is unavailable; performance will differ and no single-PC comparison is established here. |
AMD’s LLM Deployment Overview is the reference for the NPU execution modes and supported processor families. Its table is not a promise that every Ryzen AI Max+ SKU, model format or operating system supports every mode.
Verify the exact PC before installing
Record the hardware and operating system
Write down the processor model, installed system memory, Linux distribution or Windows edition, kernel or driver version, and whether the machine uses a desktop motherboard or a vendor-specific laptop design. “Ryzen AI Max+” describes a family, not one universal software target. Firmware, memory topology and the vendor’s BIOS can change what the runtime detects.
#1 Best Overall
- Built for Local AI Development: AMD Ryzen AI Halo is designed for local AI development and inference, featuring 128GB unified memory and support for up to 200B parameter models to build and run intensive AI workloads locally.
- 128GB Unified Memory: Features 128GB LPDDR5x unified memory at 8000 MT/s with 256 GB/s memory bandwidth, providing a shared memory pool across the CPU, GPU, and NPU to support larger AI models.
- AMD Ryzen AI Max+ 395 Processor: Features 16 cores, 32 threads, and Zen 5 architecture, paired with AMD Radeon 8060S integrated graphics featuring 40 RDNA 3.5 compute units and an AMD XDNA 2 NPU with up to 50 TOPS.
- Linux AI Developer Platform: Purpose-built for Linux-based AI development with full AMD ROCm software support and preloaded tools, models, and workflows optimized for local AI development.
- Compact, Connected Design: Includes a 2TB M.2 SSD, 10GbE LAN, Wi-Fi 7, Bluetooth 5.4, USB-C connectivity, and HDMI 2.1b.
Use AMD’s compatibility matrices as a gate
Open AMD’s Use ROCm on Radeon and Ryzen documentation and its linked compatibility tables. ROCm 7.2.1 documentation lists Ryzen AI Max 300 series APUs for PyTorch on Linux and Windows, but the matrix is the authority for a particular operating-system release and configuration. Do not substitute a package that merely says “AMD GPU” for one that explicitly supports your APU.
For NPU work, check the processor, model and OS entries in the Ryzen AI Software documentation before creating an environment. The Linux installation page is for Ryzen AI Software 1.8.0 and its examples currently demonstrate NPU-only execution; that is a different support statement from ROCm GPU inference.
Plan memory realistically
AMD’s ROCm Ryzen APU prerequisites page recommends 64 GB or more of system memory for development on Ryzen APUs, while explicitly noting that the requirement varies with the workload. This is guidance, not a universal minimum and not a guarantee that a particular model will load. Ryzen AI Max+ uses shared system memory rather than a fixed desktop-GPU VRAM pool, so the operating system, model weights, runtime buffers and context all compete for the same capacity.
Free tools Windows power users keep installed
One-click scans. No signup required.
- Leave headroom for the operating system and applications instead of allocating every available gigabyte to a model.
- Choose a model and quantization that fit your available memory; capacity depends on parameter count, quantization, context length and runtime overhead.
- Do not infer a single-PC model limit from AMD’s distributed demonstration. A configuration with 128 GB per machine is a documented example, not a general minimum or capacity guarantee.
Set up the GPU route with ROCm and llama.cpp
Use this route when your goal is a local LLM running primarily on the integrated GPU. AMD identifies llama.cpp as its GPU-only LLM inference framework in the Ryzen AI overview. ROCm supplies the accelerator runtime; llama.cpp supplies model loading and generation.
1. Prepare a supported software base
- Install a clean, fully updated operating system supported by the ROCm matrix for your APU. If you are on Windows, verify that the specific ROCm and workload combination is listed; do not assume Linux instructions transfer unchanged.
- Install the ROCm release specified for that matrix entry. ROCm documentation currently presents version 7.2.1, but release packages and supported kernels change, so use the current instructions rather than copying an old command from a forum.
- Reboot if the installer or driver instructions require it, then confirm that the system is using the intended AMD graphics driver before adding an inference framework.
AMD describes ROCm 7.2.1 as supporting selected Radeon GPUs and Ryzen APUs for local PyTorch development and inference. The wording “selected” matters: a successful package installation alone does not establish that your particular APU is supported.
2. Install a ROCm-enabled llama.cpp build
Obtain a build that explicitly enables the ROCm/AMD GPU backend and matches the operating system and ROCm version. AMD’s 2026 cluster article recommends prebuilt Lemonade SDK binaries for that demonstration and targets the gfx1151 architecture associated with Strix Halo and Ryzen AI Max+ 395. That advice belongs to the cited cluster configuration; check the current single-system guidance and release notes before using a binary or architecture flag.
Rank #2
- Built for Local AI Development: AMD Ryzen AI Halo is designed for local AI development and inference, featuring 128GB unified memory and support for up to 200B parameter models to build and run intensive AI workloads locally.
- 128GB Unified Memory: Features 128GB LPDDR5x unified memory at 8000 MT/s with 256 GB/s memory bandwidth, providing a shared memory pool across the CPU, GPU, and NPU to support larger AI models.
- AMD Ryzen AI Max+ 395 Processor: Features 16 cores, 32 threads, and Zen 5 architecture, paired with AMD Radeon 8060S integrated graphics featuring 40 RDNA 3.5 compute units and an AMD XDNA 2 NPU with up to 50 TOPS.
- Windows 11 Pro AI Developer Platform: Built for AI development on Windows 11 Pro with AMD ROCm software support and access to tools, models, and workflows for local AI development.
- Compact, Connected Design: Includes a 2TB M.2 SSD, 10GbE LAN, Wi-Fi 7, Bluetooth 5.4, USB-C connectivity, and HDMI 2.1b.
If you build from source, follow the current llama.cpp build instructions for the supported ROCm backend. Keep the source revision, compiler, ROCm version and GPU target together in your notes; mixing a binary built for another target is a common cause of “no devices found” or kernel-load errors.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →3. Select and stage a model
Use a model format supported by the llama.cpp build you installed, commonly a GGUF file, and store it on fast local storage. Start with a model size and quantization that leave memory headroom. The available system memory is only one constraint: context length and generation buffers grow the working set, and a model that loads with a short prompt may fail when the context is expanded.
4. Run a controlled first inference
Run the command supplied by your particular llama.cpp release, pointing it at the model file and a short prompt. A typical command-line shape is:
./llama-cli -m '/path/to/model.gguf' -p 'Give a one-sentence test response.'Use the release’s documented GPU-offload option rather than guessing a flag name; options differ between llama.cpp revisions. In the startup log, look for the ROCm/AMD backend being selected and layers being assigned to the GPU. If the log shows CPU-only execution, stop and fix the backend or compatibility issue before judging speed.
5. Increase workload gradually
- Repeat the short prompt until the backend is stable.
- Increase context length in small steps while watching memory use.
- Only then test longer conversations, batching or a larger model.
This progression distinguishes a driver or backend problem from a workload that simply exceeds shared memory. No authoritative single-PC benchmark establishes a universal token rate or model-size limit for Ryzen AI Max+, so treat your measurements as configuration-specific.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Set up the NPU route with Ryzen AI Software
Choose this route when the model and application are intended for the Ryzen AI NPU rather than the GPU. AMD’s Linux instructions for Ryzen AI Software 1.8.0 provide a concrete validation sequence. They do not install ROCm, and passing their checks does not prove that GPU inference works.
Rank #3
- Pure gaming performance with smooth 100+ FPS in the world's most popular games
- 6 Cores and 12 processing threads, based on AMD "Zen 5" architecture
- 5.4 GHz Max Boost, unlocked for overclocking, 38 MB cache, DDR5-5600 support
- For the state-of-the-art Socket AM5 platform, can support PCIe 5.0 on select motherboards
- Cooler not included
1. Confirm NPU eligibility
Check the processor and model rows in AMD’s LLM deployment table. AMD lists Ryzen AI 300 STX/KRK configurations for NPU-only and hybrid ONNX Runtime GenAI workflows. If your exact Max+ system or model is absent, use the GPU route or wait for a supported combination rather than forcing an unlisted package.
2. Install the Linux prerequisites
The Ryzen AI Software 1.8.0 Linux guide calls for Python 3.12, the Python virtual-environment module, Boost filesystem 1.74, and DKMS. On a Debian/Ubuntu-style system, the prerequisite package names are:
sudo apt install python3.12 python3.12-venv libboost-filesystem1.74.0 dkmsUse the package names and repository instructions for your distribution; if one package is unavailable, do not silently replace it with an unrelated version. The Ryzen AI page’s driver filenames and package URLs are volatile, so obtain the current AMD XRT and NPU packages from the release page rather than reusing an old download link.
3. Create an isolated Python environment
python3.12 -m venv ~/venvs/ryzen-ai
source ~/venvs/ryzen-ai/bin/activate
python -m pip install --upgrade pipInstall the Ryzen AI package and its documented dependencies inside this environment. Keep the environment dedicated to the Ryzen AI release so another project cannot downgrade its ONNX Runtime, Python or compiler dependencies.
4. Install AMD XRT and NPU components
Follow the 1.8.0 Linux page for the matching AMD XRT and NPU packages, then run the XRT environment setup script installed by those packages. The script exports the runtime paths needed by the NPU tools; open a new shell or source it again whenever you start a session outside the original installer shell.
5. Validate the driver before running a model
Run:
xrt-smi examineThe command should identify the XRT-managed device without reporting a missing driver or permission error. If it fails, check the kernel module and DKMS status, reboot after installation if requested, and verify that the package matches your kernel and processor.
Rank #4
- The Socket AM5 socket allows processor to be placed on the PCB without soldering
- Ryzen 5 product line processor for your convenience and optimal usage
- 5 nm process technology for reliable performance with maximum productivity
- Hexa-core (6 Core) processor core helps processor process data in a dependable and timely manner with maximum productivity
- 6 MB L2 plus 32 MB L3 cache memory provides excellent hit rate in short access time enabling improved system performance
6. Run AMD’s quick test
Change to the package’s quicktest directory and execute:
Recommended Free Tools
python quicktest.pyA successful quick test validates the installed NPU software path. It does not benchmark an LLM, validate ROCm, or prove that a hybrid workload is supported. Once it passes, use the model and ONNX Runtime GenAI example that AMD lists for your processor and operating system.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Understand what shared memory changes for model selection
A Ryzen AI Max+ system can present substantial system memory to the integrated GPU, but that does not behave like an independent graphics card with a guaranteed VRAM allotment. The runtime, operating system and application share the pool. Model planning should therefore be expressed as a testable budget rather than a headline capacity claim.
| Question | What to check | Decision |
|---|---|---|
| Will the weights fit? | Model parameter count, quantization and available free memory. | Start smaller or use a more compact quantization if loading fails. |
| Will the context fit? | Prompt length, requested context window and runtime buffers. | Increase context gradually; a model may load but fail at a larger context. |
| Will the accelerator be used? | llama.cpp startup output or the NPU validation tools. |
Fix backend selection before comparing responsiveness. |
| Is the result portable? | APU model, BIOS, driver, ROCm/Ryzen AI release and OS. | Record the complete configuration; another Max+ system may behave differently. |
AMD’s published 64 GB-plus guidance is a practical starting point for ROCm development, not a promise that every model fits. Conversely, a 128 GB machine can still run out of memory with an oversized model or context. Keep several gigabytes free for the desktop, file cache and runtime allocations.
Do not copy the four-node cluster settings into a normal PC
AMD’s article How to Run a One Trillion-Parameter LLM Locally: An AMD Ryzen AI Max+ Cluster Guide is a distributed demonstration, not the default installation for one computer. It documents four Framework Desktop systems, each with a Ryzen AI Max+ 395 and 128 GB of memory, running Ubuntu 24.04.3 LTS, ROCm, llama.cpp RPC and 5 Gbps Ethernet.
For that specific setup, AMD instructs users to set the integrated-GPU memory size to 512 MB in firmware and shows Linux kernel parameters:
Best Value
- AMD Ryzen 9 9950X3D Gaming and Content Creation Processor
- Max. Boost Clock : Up to 5.7 GHz; Base Clock: 4.3 GHz
- Form Factor: Desktops , Boxed Processor
- Architecture: Zen 5; Former Codename: Granite Ridge AM5
ttm.pages_limit=30720000 amdgpu.gttsize=120000The article describes checking allocation with dmesg and up to 120 GB of GTT per node for the stated configuration. These values depend on the motherboard, BIOS, kernel, driver and distributed workload. They are not ROCm or llama.cpp prerequisites for a single Ryzen AI Max+ PC. Changing them on an unrelated system can reduce stability, reserve memory unexpectedly or prevent the desktop from starting.
The article’s trillion-parameter result relies on model sharding across four nodes and its network connection. It is not evidence that one 128 GB PC can load that model, nor is it a single-machine performance benchmark.
Troubleshoot by isolating the layer that failed
The compatibility matrix does not list the exact system
Stop before installing. A neighboring APU name or a different operating system is not an equivalent entry. Use a listed configuration, or choose a CPU fallback while waiting for support.
ROCm installs but llama.cpp uses the CPU
Check that the binary was built with the ROCm backend and targets the architecture supported by your release. Read the startup log for backend and layer-offload messages. If the binary came from a different ROCm generation, replace it with a build matched to the supported matrix rather than adding random environment variables.
The model fails during loading
Reduce model size or quantization footprint, shorten the context and close memory-heavy applications. Shared-memory pressure can appear only after buffers are allocated, so a file that appears to fit on disk may still fail in memory.
xrt-smi examine cannot see the NPU
Verify Python and package versions, confirm that the AMD XRT/NPU packages match Ryzen AI Software 1.8.0, check DKMS and kernel-module errors, and source the XRT environment script in the current shell. Reboot when the installation instructions request it.
quicktest.py fails after the driver check
Run it from the package’s own quicktest directory inside the activated virtual environment. A passing xrt-smi examine proves device visibility, while the quick test exercises the installed software path; failures between the two usually indicate an environment, dependency or model-package mismatch.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Quick Recap
A repeatable setup checklist
- Identify the exact Ryzen AI Max+ processor, memory size, OS and firmware.
- Check AMD’s live ROCm and Ryzen AI compatibility matrices for that combination.
- Choose GPU ROCm plus
llama.cpp, NPU ONNX Runtime GenAI, or CPU fallback based on the supported workload. - Reserve memory headroom; AMD’s ROCm guidance starts at 64 GB or more but varies by workload.
- Install only the matching runtime and framework, keeping GPU and NPU environments separate.
- Validate the accelerator before loading a large model: inspect
llama.cppbackend logs, or runxrt-smi examinefollowed bypython quicktest.pyfor the NPU. - Increase model size, context and batching one variable at a time.
- Document the driver, runtime, model, quantization, firmware and memory settings so the result can be reproduced.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
