Olmo-core 3 is Ai2’s redesigned open training system for mixture-of-experts (MoE) language models. Ai2 reports higher throughput in specific tests and systems benchmarks at trillion-parameter scale, but those results are not evidence that it has trained a high-quality trillion-parameter model.
What is Olmo-core 3?
Olmo-core is Ai2’s open framework for developing and training models in the OLMo ecosystem. Olmo-core 3 is a reworked training stack aimed at large, sparse MoEs: models with many total parameters that use only a subset of their experts for each token. Ai2 describes the release as core infrastructure for the next generation of OLMo and as a system outside researchers and developers can adapt for their own MoE training and experiments.
As an Amazon Associate I earn from qualifying purchases.
The engineering challenge is that sparse computation does not make the whole system small. Model state still occupies GPU memory, while routing tokens to experts distributed across a cluster requires communication and coordination. Those costs can reduce the practical advantage of using only part of a model for each token.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →How does Olmo-core 3 change MoE training?
From gathering weights to keeping experts resident
Ai2 says its earlier MoE implementation used fully sharded data parallelism (FSDP) configured to gather and then reshard weights for every small batch. Olmo-core 3 instead uses a distributed data parallelism (DDP)-based design that keeps experts resident on GPUs and routes data to them. The change targets the overhead of repeatedly moving model weights, while making expert placement and token movement central systems concerns.
#1 Best Overall
A combination of parallelism and GPU optimizations
- Expert parallelism distributes experts across GPUs.
- Pipeline parallelism splits model layers across groups of GPUs.
- A distributed optimizer spreads optimizer state across GPUs.
- Rowwise expert parallelism places routed data directly into expert input buffers.
- GPU-resident routing keeps routing metadata on GPUs instead of copying it to the CPU.
- Grouped GEMM combines small expert computations to use GPU execution more efficiently.
- MXFP8 support uses a lower-precision format where reduced compute or data movement is sufficient to outweigh conversion overhead.
These are interdependent design choices, not a menu of guaranteed speedups. Less data movement may come with conversion work; faster computation can increase communication pressure. Ai2 reports that some apparently promising changes did not improve end-to-end results, including attempts to overlap communication and computation that instead slowed execution.
What performance did Ai2 report?
The figures below come from Ai2’s October 1, 2026 announcement. They describe particular benchmarks and tests, not independent replications or universal performance guarantees.
Rank #2
| Test | Ai2’s reported result | What the result establishes |
|---|---|---|
| Expert-pool scaling | Expanded the pool from 8 to 128 experts, selecting four per token and using about 3.2B active parameters per token. Total parameter capacity rose from 4.6B to 47B while throughput declined by less than 5%. | A release benchmark of scaling expert capacity; it does not establish that every workload or hardware configuration will have the same throughput behavior. |
| Comparison with the earlier implementation | 52,000 tokens per second per GPU versus 19,400, about 2.7×. Ai2 calls this a preliminary test of a 47-billion-parameter MoE on eight NVIDIA B300 GPUs. | A preliminary result on the stated model and hardware, rather than a general speedup claim. |
| MXFP8 versus BF16 | About 21% higher training throughput with MXFP8; peak active memory fell from 103 GiB to 95 GiB. The controlled benchmark used four NVIDIA B300 GPUs, uniform work distribution across experts, and MXFP8 where Ai2 found it most helpful. | A result under a specified controlled setup. It does not show that MXFP8 will improve every workload, since conversion overhead can offset its benefits. |
| Trillion-parameter systems test | 1.2 trillion total parameters and 58.36 billion active parameters per token across 512 GPUs, with highest observed throughput of 858 TFLOP/s/GPU. | Ai2 used random routing to measure systems performance, not the quality of a trained model. |
| Maximum stated capacity | 2.38 trillion total parameters. | Ai2 describes this as a short-capacity test with DeepEP v2, not a full training run or sustained training-throughput result. |
For evaluating the numbers, keep total parameter capacity separate from active parameters per token, and distinguish a systems test from a full training run. A trillion-parameter configuration demonstrates that the system reached that scale under the stated test conditions; it does not show that Ai2 trained a high-quality language model of that size.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →What engineering lessons did Ai2 report?
The announcement also describes failure modes that matter when interpreting throughput work. Ai2 calls one routing-balancing problem “token gerrymandering”: a score intended to encourage balanced routing improved even as actual workload balance worsened. Tests lowering experts’ learning rates did not improve results. Ai2 also observed cases where computation time depended on input values despite identical matrix shapes. Together with the communication-overlap slowdown, these examples show why local optimization metrics may not predict end-to-end training performance.
How can researchers try Olmo-core?
Ai2’s public Olmo-core repository describes the project as “PyTorch building blocks for the OLMo ecosystem.” It recommends installing from source for development and lists the PyPI package name as ai2-olmo-core. The project is licensed under Apache-2.0. The repository also notes that some features need optional dependencies, including attention backends, float8 training, and dropless MoE.
The repository provides official training scripts for OLMo 2 and OLMo 3, with launches documented through torchrun or Ai2’s Beaker CLI where available. Ai2’s published Docker images include core and optional dependencies but do not install Olmo-core itself. The README cautions that those images may not work on clusters with different hardware or driver/CUDA versions, so users should check compatibility and install the project as directed rather than assume that the image alone supplies a ready-to-run environment.
Rank #4
- Built for Local AI Development: AMD Ryzen AI Halo is designed for local AI development and inference, featuring 128GB unified memory and support for up to 200B parameter models to build and run intensive AI workloads locally.
- 128GB Unified Memory: Features 128GB LPDDR5x unified memory at 8000 MT/s with 256 GB/s memory bandwidth, providing a shared memory pool across the CPU, GPU, and NPU to support larger AI models.
- AMD Ryzen AI Max+ 395 Processor: Features 16 cores, 32 threads, and Zen 5 architecture, paired with AMD Radeon 8060S integrated graphics featuring 40 RDNA 3.5 compute units and an AMD XDNA 2 NPU with up to 50 TOPS.
- Linux AI Developer Platform: Purpose-built for Linux-based AI development with full AMD ROCm software support and preloaded tools, models, and workflows optimized for local AI development.
- Compact, Connected Design: Includes a 2TB M.2 SSD, 10GbE LAN, Wi-Fi 7, Bluetooth 5.4, USB-C connectivity, and HDMI 2.1b.
What Olmo-core 3 does—and does not—claim
Ai2’s stated goal is to “scale MoE training into the trillion-parameter range while preserving computational efficiency.” The release presents a redesigned open training stack, implementation details, and benchmark results supporting that goal. Its largest-scale figures are systems measurements, including random-routing and short-capacity tests; they are not evidence of trained-model quality at trillion-parameter scale. Practical results will depend on the workload, hardware, software environment, and how communication and computation interact.
Free tools Windows power users keep installed
One-click scans. No signup required.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
