Recommended Free Tools
In one Qwen3-14B decentralized-exchange benchmark, DAPO led the no-think holdout with a 0.6467 native score and 39 successful episodes out of 40. With Thinking-2048 enabled, DAPO-refill led at 0.7105 and 38/40. But the untrained thinking-enabled model scored 0.6830, ahead of every trained thinking configuration except DAPO-refill. The result is a useful warning against choosing a reinforcement-learning trainer by its training curve alone—and not a universal ranking of algorithms.
What the benchmark measured
The benchmark article, published October 6, 2026, tested GRPO, DAPO, CISPO and several author-defined configurations on a deterministic decentralized-exchange (DEX) arbitrage task using Qwen3-14B. It evaluated each available configuration in two reasoning regimes: no-think and Thinking-2048.
As an Amazon Associate I earn from qualifying purchases.
Each episode gave the agent a frozen snapshot of automated market-maker pools. The agent had to find a profitable cycle through two to four pools and choose an input amount that remained profitable after fees, gas and slippage. A simulated Ethereum Virtual Machine (EVM) verifier checked whether the proposed execution worked.
The article evaluated final step-50 checkpoints on a frozen holdout: five task families, two held-out seeds and four samples per task, for 40 episodes across 10 distinct unseen scenarios. It also evaluated the untrained model in both reasoning regimes. That baseline matters: a trained model’s score is more informative when compared with what the same model could do before reinforcement learning.
#1 Best Overall
Which configuration scored highest on unseen tasks?
The table gives the article’s native holdout score and successful-episode count. These are holdout results, not the composite reward used during training. Every listed result is from this benchmark’s setup and 40-episode evaluation; it should not be read as a general performance estimate for the algorithm.
| Configuration | No-think native holdout | Thinking-2048 native holdout |
|---|---|---|
| Untrained baseline | 0.3856 (16/40) | 0.6830 (37/40) |
| GRPO | 0.6424 (38/40) | 0.6798 (38/40) |
| DAPO | 0.6467 (39/40) | 0.6703 (38/40) |
| DAPO-refill | 0.5839 (34/40) | 0.7105 (38/40) |
| GDPO | 0.6245 (36/40) | 0.6602 (35/40) |
| CISPO | 0.6190 (36/40) | 0.6068 (33/40) |
| DAPO + ADAPO | 0.6332 (37/40) | 0.6250 (34/40) |
| DAPO + REPO-R | 0.5724 (31/40) | 0.6461 (34/40) |
| CISPO + REPO-R w5 | Not reported | 0.6243 (34/40) |
No-think: DAPO had the highest reported score
DAPO reached 0.6467 with 39 successful episodes, narrowly ahead of GRPO’s 0.6424 and 38/40. That is the top no-think result in the table, but the score gap is small and the evaluation comprises only 40 episodes. Without confidence intervals or repeated independent training runs, this result does not establish that DAPO will reliably outperform GRPO in other tasks or runs.
Thinking-2048: DAPO-refill led, but the untrained baseline was already strong
DAPO-refill achieved 0.7105 and 38/40, the highest reported thinking-mode score. The untrained model scored 0.6830 and succeeded in 37/40. DAPO-refill is the only trained thinking configuration in the table to exceed that untrained score. GRPO and DAPO each recorded 38/40 successes yet scored below the baseline, illustrating why counting successful episodes alone can hide differences in solution quality.
Why the training reward does not settle the comparison
The article’s training reward combines three terms: native task score, an execution-efficiency reward weighted by 0.05, and a reliability reward weighted by 0.20. The holdout table reports native score alone. These measures answer different questions, so a higher training reward cannot be treated as a higher unseen-task score.
CISPO makes the distinction particularly clear in the reported Thinking-2048 results: its last-10 training reward mean was 0.8951, while its native holdout score was 0.6068. Those figures are not directly comparable because they are different metrics, but together they show why the training curve by itself cannot demonstrate generalization. The untrained thinking baseline’s 0.6830 holdout score also exceeded CISPO’s result.
What the algorithm names mean here
GRPO
Group Relative Policy Optimization estimates a baseline from the scores of a group of sampled outputs rather than relying on a separate critic model. It was introduced in DeepSeekMath as a PPO variant.
Rank #3
DAPO
Decoupled Clip and Dynamic Sampling Policy Optimization is associated with techniques including Clip-Higher, dynamic sampling, token-level policy-gradient loss and overlong-reward shaping. The benchmark’s DAPO result reflects its tested implementation and configuration.
Free tools Windows power users keep installed
One-click scans. No signup required.
CISPO
Clipped Importance Sampling Policy Optimization clips importance-sampling weights as detached coefficients, rather than applying PPO-style hard clipping to the ratio. That distinction describes the method; it does not predict its ranking on this particular holdout.
GDPO and named combinations
Group reward-Decoupled Normalization Policy Optimization uses decoupled normalization for multi-reward optimization. DAPO-refill, ADAPO, REPO-R, and the combined configurations in the table are the benchmark author’s tested setups. Their labels should not be taken to mean that every implementation of the named methods uses the same configuration.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How much training compute did the reported setup use?
The benchmark used a fixed seed of 42, a learning rate of 1e-5 with linear decay, 50 optimizer steps, group size 4, eight completions per fresh rollout batch and two policy updates per rollout. It reports Qwen3-14B revision 40c06982…, NF4 quantization with BF16 compute, LoRA rank 32 and alpha 64, on one NVIDIA RTX PRO 6000 Blackwell GPU with 96 GB of memory.
On that hardware and setup, the article reports 12.3–14.1 GPU-hours for 50 Thinking-2048 steps, compared with 2.2–3.1 GPU-hours for 50 no-think steps. Thinking-mode steps took 4.5–5.9 times longer, and listed Thinking-2048 runs reached 66.5–71.8 GiB peak allocated GPU memory. These are the benchmark’s reported measurements, not general cost or memory requirements for training these algorithms.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Quick Recap
How to interpret the result
- There is no single winner across both reasoning regimes. DAPO led the no-think table; DAPO-refill led Thinking-2048.
- Thinking did not automatically improve trained-model generalization. In the thinking condition, the untrained baseline outscored every trained configuration except DAPO-refill.
- Success counts and scores capture different things. GRPO and DAPO each had 38/40 thinking-mode successes, but their scores were below the 37/40 untrained baseline.
- The holdout is informative but limited. Forty episodes over 10 unseen scenarios offer a more direct generalization check than training reward alone, but the article reports no confidence intervals or repeated independent training runs. Small differences should not be treated as proof of broad superiority.
- Compute is part of the choice. In this setup, Thinking-2048 took substantially more reported GPU time, yet most trained configurations did not beat the untrained thinking baseline on native holdout score.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
