Windows ML is Microsoft’s Windows-supported framework for running machine-learning inference inside a Windows application. It is powered by ONNX Runtime: an app supplies an already-trained model, sends it inputs, and receives predictions locally. CPU execution is the baseline; compatible execution providers can route work to a GPU or NPU.
That makes Windows ML a developer platform, not a consumer AI feature and not a guarantee that every model will run on every PC. The model may need conversion to ONNX, and acceleration depends on the Windows release, hardware, drivers and provider support.
As an Amazon Associate I earn from qualifying purchases.
What Windows ML does—and what it does not do
Windows ML gives Windows developers a supported way to embed local inference in an application. “Inference” means applying a trained model to new input: classifying an image, detecting objects in video, transcribing audio or generating another output. Training the model, converting it, and optimizing it are separate activities that happen before the app performs inference.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
The standard Windows ML path is built around ONNX, the Open Neural Network Exchange format. Models created in workflows involving PyTorch, TensorFlow/Keras, TFLite or scikit-learn can be used when they are exported or converted to a compatible ONNX model. Some applications may use ONNX Runtime (ORT) model files as well. A model that remains only in its original framework is not automatically ready for the Windows ML ONNX workflow.
#1 Best Overall
- NVIDIA Volta GV100 Architecture — 4,608 CUDA Cores, 640 1st-Gen Tensor Cores delivering 14 TFLOPS FP32 and 112 TFLOPS deep learning performance for AI training, inference, HPC, and scientific computing workloads
- 32GB HBM2 ECC Memory — 900 GB/s Bandwidth — High-bandwidth memory on a 4096-bit bus with ECC error correction provides the memory capacity and throughput required for the largest AI models, simulations, and datasets
- PCIe 3.0 x16 Interface — 250W TDP — Standard PCIe Gen3 connectivity with passive cooling designed for enterprise rack server deployment in HPE ProLiant, Dell PowerEdge, and Supermicro platforms with adequate chassis airflow
- NVLink — Scale to 96GB Unified Memory — Connect two V100 GPUs via NVLink at 300 GB/s bi-directional bandwidth to scale GPU memory from 32GB to 96GB for larger AI training and HPC workloads
- Multi-Precision Computing — Supports FP64 (7 TFLOPS), FP32 (14 TFLOPS), FP16 (112 TFLOPS) and INT8 precision modes for flexible deployment across training, inference, and scientific simulation workloads
Windows ML is therefore best understood as an integration layer: it exposes ONNX Runtime APIs to the app and helps select the execution provider that can run the model on the available hardware.
How an app runs a model with Windows ML
The exact code differs by language and app architecture, but the runtime sequence is consistent.
- Obtain or convert the model. Start with an ONNX model, or export a model from its training framework to ONNX. Check that its operators, input shapes and data types are supported by the intended runtime and provider.
- Install the runtime and app dependencies. Choose the Windows ML packages and the deployment style appropriate for the app. The language-specific setup determines the required Windows App SDK and language runtime components.
- Load the model through ONNX Runtime APIs. The application creates an inference session, reads the model and prepares the input and output tensors. The app, rather than Windows ML, remains responsible for application-level preprocessing such as resizing an image or normalizing audio.
- Select an execution provider. CPU is the documented starting point. If the target device and driver support another provider, the app can acquire and configure it so compatible parts of the graph execute on a GPU or NPU.
- Prepare inputs in the model’s expected format. Names, dimensions, tensor types and ordering must match the exported model. A mismatch here can fail before any hardware acceleration is attempted.
- Run inference and consume outputs. The runtime executes the graph and returns tensors. The app turns those tensors into a user-visible result, such as a label, bounding box, score or generated text.
- Handle fallback and errors. Provider support is not universal. A robust application checks whether the chosen provider is present and compatible, and keeps a CPU path for devices that cannot use the preferred accelerator.
This flow runs an existing model; it does not train one. Training and model conversion normally occur in a separate development pipeline.
Free tools Windows power users keep installed
One-click scans. No signup required.
CPU, GPU and NPU: which execution target fits?
Microsoft describes the three hardware classes by workload and operating characteristics rather than by a universal speed ranking. Actual performance depends on the model, device, provider, driver and measurement conditions.
| Execution target | Role described by Microsoft | Important qualification |
|---|---|---|
| CPU | Universal fallback and documented starting path | Available as the basic route when a specialized provider is unavailable or incompatible. |
| GPU | High-throughput workloads such as image, video and generative AI | Requires a compatible GPU, driver and execution provider; throughput varies by model and configuration. |
| NPU | Sustained, battery-efficient on-device inference | Requires an NPU and a provider that supports the device and model; efficiency does not imply a fixed performance advantage. |
For a desktop application processing large batches of images, a GPU provider may be the practical target. A battery-powered PC running a small model continuously may benefit from an NPU provider. A broad Windows application with unknown hardware should retain CPU inference as its compatibility baseline. These are workload decisions, not blanket claims that one processor is always faster.
Rank #2
- Delivers 25 TFLOPS of AI inference performance using four MemryX MX3 accelerators in a compact M.2 2280 M-key module.
- Connects through PCI Express Gen 3 and above for straightforward integration into embedded and edge systems with an M.2 M-key slot.
- Includes 42 MB of distributed on-chip weight storage to support low-latency, batch-1 pipelined inference.
- Designed for low-power deployment with a 1 W–12 W operating range and host-managed airflow requirements.
- Supports ONNX, TensorFlow, Keras, and TFLite models through the MemryX software stack on Linux and Windows hosts.
Execution providers and hardware compatibility
In ONNX Runtime, an execution provider connects the runtime to a hardware backend. Windows ML includes CPU and DirectML providers. Other providers in Microsoft’s catalog are acquired separately rather than included in the base runtime.
The current 2.x catalog lists hardware-optimized options including AMD MIGraphX, AMD VitisAI, NVIDIA NvTensorRtRtx, Intel OpenVINO, Qualcomm QNN and an experimental WebGPU provider. The list and support status are version-sensitive. A provider may require a particular vendor driver, device generation, SDK, license or model operator set, so check the provider’s current release requirements before committing to it.
Microsoft’s provider documentation describes these catalog-based hardware-optimized providers as available on Windows 11 version 24H2, build 26100, or later, subject to compatible hardware and drivers. That condition applies to the specialized providers acquired through the catalog; it should not be confused with the minimum Windows version an application can target through the Windows App SDK. An app may support a wider Windows range while using CPU inference, then enable an accelerator only when the machine meets the provider’s requirements.
Why provider selection can fail
- The PC lacks the required GPU, NPU or driver version.
- The installed Windows release is older than the provider’s requirement.
- The model contains operators or data types that the provider does not implement.
- The provider package is not installed, is the wrong architecture, or is incompatible with the app’s runtime.
- The provider is experimental or has support restrictions that make it unsuitable for a production deployment.
Design provider selection as a capability check, not as a hard-coded assumption. If the preferred provider cannot be created or cannot execute the model, report the reason and use the CPU path when acceptable.
Windows and language prerequisites
Windows ML applications can target Windows versions supported by the Windows App SDK. The documented getting-started path begins with CPU inference, which is the least hardware-specific route.
Rank #3
- ✅Powered by 26 Tera-Operations Per Second (TOPS) Hailo-8 AI Processor. 2.5W typical power consumption
- ✅Scalable, enabling simultaneous processing of multi-streams & multi-models
- ✅Enabling real-time, low latency and high-efficiency AI inferencing on the edge devices
- ✅Supports TensorFlow, TensorFlow Lite, ONNX, Keras, Pytorch frameworks
- ✅Supports Linux and Windows. Supports the temperature range of -40°C to 85°C
For the complete Windows ML API surface, Microsoft’s current getting-started and deployment guidance specifies .NET 8 or later for .NET applications. The corresponding setup paths document C++20 and Python 3.10 through 3.13 requirements. A project should verify the exact package, target framework and language prerequisites it plans to ship, because those details can change with Windows ML and Windows App SDK releases.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesThese language requirements are separate from accelerator requirements. Meeting the .NET, C++ or Python prerequisite does not provide a GPU or NPU provider; the hardware, Windows build and driver still have to satisfy that provider’s conditions.
Self-contained versus framework-dependent deployment
Microsoft documents two ways to ship the Windows ML runtime with an application.
| Deployment approach | How it works | Trade-off |
|---|---|---|
| Self-contained | The app bundles the Windows ML runtime, including the Windows ML API, ONNX Runtime engine and DirectML provider. | The package is larger, but it does not depend on a separately installed runtime and the developer controls when runtime updates are shipped. |
| Framework-dependent | The app uses a shared Windows App SDK runtime installed on the system. | The bundle is smaller and receives runtime servicing updates, but installation depends on that shared runtime being present. |
Microsoft estimates the self-contained Windows ML runtime binaries at approximately 41 MB. That is an approximate Microsoft figure for the listed runtime components, not an independent measurement of a complete application installer. Vendor execution providers are not included in that runtime estimate; they must be acquired separately or bundled by the developer.
Choose self-contained deployment when predictable runtime control and independence from machine-wide installation matter more than package size. Choose framework-dependent deployment when reducing the app download and receiving shared runtime servicing are more important, and your installer can reliably establish the Windows App SDK dependency.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Rank #4
- 48GB AI graphics accelerator
Windows ML and DirectML are related, not interchangeable
DirectML is a lower-level machine-learning API built around Direct3D 12. Windows ML uses ONNX Runtime APIs and provides a higher-level route for Windows ONNX deployments, including management of hardware-specific execution providers.
Microsoft’s DirectML overview says: “DirectML is in sustained engineering. DirectML continues to be supported, but new feature development has moved to Windows ML for Windows-based ONNX Runtime deployments.” In practical terms, existing DirectML-based software is not automatically obsolete, and DirectML has not been discontinued. For a new Windows application that needs ONNX Runtime integration and provider management, Windows ML is the direction Microsoft documents for new feature development.
Privacy: local inference is not automatically offline
When Windows ML runs inference on the device, the model computation can occur locally. That fact alone does not prove that an application is fully offline or that its inputs never leave the PC.
An app may download the model or an execution provider, contact a service for unrelated features, send telemetry, or apply its own data-retention policy. To make a privacy claim, inspect the application’s network behavior and privacy documentation, and determine where models, prompts, images, audio or telemetry are sent. Windows ML supplies a local inference capability; it does not impose one universal data-handling policy on every app that uses it.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchHow to evaluate performance without misleading benchmarks
There is no universal CPU-versus-GPU-versus-NPU number that applies to all Windows PCs. A meaningful test must identify the exact model, device, execution provider, driver, input dimensions, batch size, precision, power mode and measurement method. Results from one configuration should not be transferred to another PC or model.
Best Value
- Professional AI & Creator Workstation: AMD Radeon AI PRO R9700 GPU with 32GB GDDR6 is engineered for AI development, professional content creation, and compute-intensive workloads.
- Massive 32GB Memory Capacity: 32GB of GDDR6 memory on a 256-bit bus provides ample bandwidth for large AI models, 8K video editing, and complex 3D rendering.
- Advanced RDNA 4 with AI Accelerators: 64 Compute Units with 3rd Gen Ray Tracing and dedicated 2nd Gen AI Accelerators for groundbreaking AI performance and visual computing.
- Professional Blower Cooling: Efficient single blower design exhausts heat directly out of the chassis, ideal for multi-GPU workstation and server configurations.
- Enterprise-Grade Thermal Solution: Vapor chamber heatsink with industrial Honeywell PTM7950 thermal interface material ensures reliable cooling under sustained professional loads.
Measure the part of the application users actually feel: startup and model-load time, first-result latency, sustained throughput, memory use and battery impact. Test the CPU fallback separately, because a provider that is fast on one model may not support another model’s operators and may cause a fallback or failure.
A practical preflight checklist
Before shipping a Windows ML feature, verify each item below on the actual target devices:
- The model is exported to ONNX or another format supported by the chosen ONNX Runtime path.
- Input names, dimensions, data types and output handling match the model.
- The app’s Windows App SDK and language prerequisites are satisfied.
- CPU inference works independently of optional accelerators.
- Each accelerator provider is installed, licensed where applicable and compatible with the Windows build, device and driver.
- For catalog-based hardware-optimized providers, the machine meets the Windows 11 24H2 (build 26100) or later condition documented by Microsoft.
- Provider creation and model execution failures produce a controlled fallback or a useful diagnostic.
- Installer tests cover both self-contained and framework-dependent assumptions, including machines without the shared runtime.
- Privacy documentation describes downloads, network calls, telemetry and data retention rather than relying only on the word “local.”
When Windows ML is the right choice
Windows ML is a strong fit when a Windows application needs on-device inference, wants ONNX Runtime compatibility, and must adapt to a range of Windows hardware. It lets the same application start with CPU execution and add GPU or NPU providers where the device and model support them.
It is not a magic compatibility layer. You still need a supported model format, a tested provider configuration and a deployment plan. If a model cannot be converted to ONNX, uses unsupported operators, or depends on a provider unavailable on the target machines, Windows ML cannot remove those constraints. Treat the framework as the Windows-supported runtime foundation, then validate the model and hardware combination you intend to ship.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
