October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsClean PCRecommendedOne scan can reveal what keeps slowing WindowsLook for cleanup and repair opportunities.Run ScanOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
Redmond desk6 min

Microsoft Adds Experimental GGUF Support and New APIs to Windows ML

Windows ML is production-ready, but Microsoft’s new GGUF integration is experimental and its Windows-native Runtime API is in preview. Here’s how the APIs, model formats and hardware requirements differ.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Microsoft’s October 7, 2026 Windows ML announcement adds an experimental route for running GGUF models with llama.cpp, introduces task-specific text generation and speech recognition APIs, and previews a lower-level Windows-native Runtime API. The distinction matters: Windows ML itself has been generally available for production since September 2025, but the newly announced GGUF integration is experimental and the Runtime API is in preview.

What is Windows ML?

Windows ML is Microsoft’s framework for running AI models locally on Windows. It is powered by ONNX Runtime and connects models to hardware-specific execution providers, which can use a CPU, GPU or NPU when the device and provider support that route. Microsoft says Windows ML supports x64 and ARM64 systems and models originating from frameworks including PyTorch, TensorFlow/Keras, TFLite and scikit-learn.

As an Amazon Associate I earn from qualifying purchases.

Windows ML became generally available on September 23, 2025, as part of Windows App SDK 1.8.1. That release announcement described support for Windows 11 version 24H2 or newer. For present-day project requirements, Microsoft Learn is the more current reference: it specifies a Windows version supported by the Windows App SDK, and says optimized providers for NPUs and specific GPU hardware require Windows 11 version 24H2 (build 26100) or newer. CPU and GPU inference through DirectML are available on supported Windows versions; the exact available acceleration depends on the system and provider.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

What’s new in Windows ML?

The October announcement adds three developer-facing routes: task-specific APIs for generation and speech recognition, experimental llama.cpp support for GGUF models through the text-generation route, and a preview of a Windows-native Runtime API for more explicit control over data and pipeline execution. The existing ONNX Runtime APIs remain available alongside the new Runtime API.

#1 Best Overall
Route Model format or input Control level Status
Text Generation API GGUF or ONNX language model Task-specific; Windows ML selects an execution engine, including llama.cpp for GGUF Announced October 7, 2026; the llama.cpp integration is experimental (Microsoft)
Speech Recognition API Audio transcribed with an ONNX Whisper model Task-specific transcription Announced October 7, 2026 (Microsoft)
Windows-native Runtime API Windows-native image, video, audio and text types Lower-level pipeline composition, data handling and device placement Preview (Microsoft)
Existing ONNX Runtime APIs ONNX models Existing ONNX Runtime development path Supported alongside the new Runtime API (Microsoft)

How do I run a GGUF model on Windows ML?

The announced path is to use the Windows ML Text Generation API with a GGUF language model, including models from Hugging Face. Windows ML chooses an execution engine for the model; Microsoft says the experimental integration uses llama.cpp for GGUF. The announcement describes a development API, not a guarantee that every GGUF model or Windows device will work identically.

  1. Check your Windows and app prerequisites. Use a Windows version supported by the Windows App SDK. For optimized NPU or specific GPU providers, Microsoft Learn lists Windows 11 version 24H2 (build 26100) or newer.
  2. Choose the model route. Use the Text Generation API for a GGUF model, or use an ONNX language model if that is the format you have. The experimental llama.cpp integration applies to the GGUF route.
  3. Choose hardware through a supported provider. Windows ML can access CPU, GPU or NPU execution providers, but the actual path depends on the hardware, installed provider and model support.
  4. Prototype locally if useful. Microsoft says an OpenAI-compatible endpoint is available for local prototyping with the OpenAI SDK. That makes it possible to try a local model through a familiar client interface; it does not make the model a cloud service.

Microsoft describes local inference as a way to reduce latency, keep workload data on the device and avoid per-token cloud inference charges. These are potential benefits, not guaranteed results: model size, provider support, hardware and workload all affect performance and practicality.

What can the task-specific APIs do?

Generate text with a local language model

The Text Generation API accepts GGUF and ONNX language models. Its value is a task-oriented interface that can choose an execution engine rather than requiring the developer to build every inference detail around a particular backend. GGUF support through llama.cpp is experimental; the broader Windows ML production status does not change that maturity label.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Transcribe audio with an ONNX Whisper model

The Speech Recognition API transcribes audio using an ONNX Whisper model. Microsoft says developers can chain the APIs—for example, transcribe speech and pass the resulting text to a GGUF model. That enables a local speech-to-text-to-generation workflow, subject to the chosen models and device’s capabilities.

Rank #2
Dell Latitude 5420 14" FHD Business Laptop Computer, Intel Quad-Core i5-1145G7, 16GB DDR4 RAM, 256GB SSD, Camera, HDMI, Windows 11 Pro (Renewed)
  • 256 GB SSD of storage.
  • Multitasking is easy with 16GB of RAM
  • Equipped with a blazing fast Core i5 2.00 GHz processor.

When would a developer use the Windows-native Runtime API?

The Runtime API preview is aimed at developers who need more control than a task-specific call provides. Microsoft describes direct use of Windows-native image, video, audio and text types through zero-copy paths; deterministic multi-model pipelines with explicit CPU, GPU or NPU placement at each stage; and ahead-of-time model loading and compilation.

Those features target data movement, pipeline composition and predictable placement across stages. They are not a reason to migrate an existing ONNX Runtime application by default: Microsoft says ONNX Runtime APIs remain supported. Developers can choose the higher-level task APIs for straightforward generation or transcription, and evaluate the preview Runtime API when explicit multi-stage orchestration or native data handling is important.

Can Windows ML run models on a GPU, NPU or CPU?

Yes, Windows ML is designed to use supported CPU, GPU and NPU execution providers, but it does not make every model run on every processor or guarantee that one processor class is always fastest. Microsoft Learn says CPU and GPU inference through DirectML are available on supported Windows versions. Optimized providers for NPUs and specific GPU hardware require Windows 11 version 24H2 (build 26100) or newer. Model compatibility, device hardware and provider availability determine the usable path and its performance.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  • CPU: A broadly available fallback where a compatible provider and model are present; do not assume it will match an accelerator’s throughput.
  • GPU: Can use supported GPU providers, including DirectML on supported Windows versions. The relevant provider and model support matter more than the generic label “GPU.”
  • NPU: An option on supported hardware; Microsoft’s stated optimized-provider requirement is Windows 11 version 24H2 (build 26100) or newer.

Windows ML is not limited to Copilot+ PCs or the new RTX Spark systems. Microsoft Learn lists x64 and ARM64 Windows PCs and CPU, GPU and NPU execution providers; the exact compatible combinations vary.

Rank #3
15.6 Inch Win 11 Laptop Computer, N4020, 4GB DDR4 RAM, 128GB Storage | LPDDR4 RAM, 180° Lay-Flat, Dual-Band WiFi, Type-C PD Fast Charging, Thin & Lightweight for Students, Home & Office, Gray
  • WINDOWS 11 | STABLE PERFORMANCE: Powered by Intel Celeron N4020 processor and Windows 11 system, this laptop delivers stable performance for everyday computing tasks. It supports web browsing, online learning, document editing, email communication, and basic office work with optimized power efficiency, providing a practical and reliable experience for essential daily use for daily use.
  • 15.6” FHD IPS DISPLAY: Features a 15.6-inch Full HD IPS display with narrow bezels, offering wider viewing angles and clearer image details compared to standard panels. The improved screen-to-body ratio enhances visual experience for study, reading, document work, and video playback, making it suitable for both productivity and entertainment use.
  • 4GB DDR4 + 128GB eMMC STORAGE: Equipped with 4GB DDR4 memory and 128GB eMMC storage for everyday basics such as browsing, documents, email, and online learning platforms. The built-in TF card slot supports storage expansion up to 1TB, giving you more flexibility for files, photos, videos, and daily documents. TF card not included.
  • CONNECTIVITY & PORTS: Includes 1× TF card slot, 2× USB 3.2 Gen1 ports, and 2× full-featured Type-C ports (USB 3.2 Gen1). The Type-C ports support data transfer, charging, and video output, enabling flexible connection with external devices such as monitors, storage, and peripherals for daily work and study use.
  • LIGHTWEIGHT DESIGN | ONLINE COMMUNICATION: Designed with a slim, portable profile, this laptop is easy to carry for school, commuting, and travel. A built-in 1MP front camera supports online classes, video meetings, remote communication, and everyday conferencing. The 3300mAh battery works with the low-power system design to support practical daily use, while thermal optimization helps maintain quieter operation during extended tasks.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

How does the update fit into Microsoft’s wider Windows AI stack?

Microsoft’s October 7 developer post also describes updates to the surrounding open-source development stack. It says PyTorch offers official native Windows Arm64 CPU builds, NVIDIA publishes CUDA-enabled Windows Arm64 packages for supported hardware, and the Windows Triton distribution brings triton.jit, torch.compile and custom GPU kernels to supported Windows GPUs. The post demonstrates exporting a PyTorch model graph to ONNX for deployment. That example shows a development workflow; it is not evidence of a general performance gain.

In a parallel October 7 Windows announcement, Microsoft framed its strategy as “hybrid intelligence”: local models when appropriate and cloud services when needed. It reported more than 2 trillion local inferences per month across Copilot+ PCs. That is Microsoft’s figure, not an independent measurement, and it describes Copilot+ PC usage rather than a Windows ML benchmark.

Microsoft also named RTX Spark PCs as hardware for demanding local AI work and said Surface Laptop Ultra supports up to 128 GB of unified memory and local execution of models exceeding 120 billion parameters. These are vendor-stated hardware capabilities, not Windows ML requirements. The same Windows announcement said related Copilot features were expected to reach Copilot+ PCs over the coming months; that was a planned rollout stated on October 7, 2026, not confirmation that the rollout has since completed.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Microsoft’s October 7 developer announcement describes its contributions with NVIDIA and the wider llama.cpp community, including CUDA kernel optimization, kernel fusion, CPU–GPU scheduling, weight repacking, CUDA graphs, speculative decoding methods, multi-GPU execution, NVFP4, additional architectures and backend sampling. These are descriptions of project work, not independent benchmark results. The announcement does not establish that every optimization applies to every model, device or Windows ML workflow.

What should developers take away?

Windows ML’s production-ready foundation now sits alongside newer, less mature ways to build local AI features: task-specific APIs, experimental GGUF support through llama.cpp, and a Runtime API preview for explicit control of data and multi-model execution. The practical choice depends on the format and control the project needs. Check Windows and provider requirements, test the actual model-device combination, and treat Microsoft’s performance and local-inference benefits as possibilities to validate for the target workload.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Outdated Drivers Are Slowing You DownFree scan - exact matches
Windows Errors? Fix Them Before They SpreadFree repair scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.