Recommended Free Tools
In the original 52-scenario benchmark, an AI agent using Radar’s Kubernetes MCP tools reported a slightly higher diagnosis pass rate and lower tool, token, and time totals than the same model using raw kubectl. A later, different 54-scenario rerun found a much smaller tool-call reduction than the original headline suggested, while reporting a faster median time to correct diagnosis. These are vendor-published results from fault-injection tasks—not independent proof that MCP improves agents, or that either setup is safer or better at fixing production clusters.
What the original 52-scenario benchmark compared
Daria Dovzhikova’s July 21, 2026 report describes 52 fault-injection scenarios on a live Amazon EKS cluster, including crash loops, misconfigurations, resource pressure, broken rollouts, and failures whose cause was separated from the visible symptom. The benchmark held prompts and success criteria constant while comparing Claude Sonnet 4.6 in two setups: an agent with a shell and raw kubectl, and the agent connected to Radar’s Kubernetes MCP server. Success meant identifying the actual root cause. Dovzhikova disclosed a connection to Radar, and the report was originally published at RadarHQ. Read the original 52-scenario write-up.
As an Amazon Associate I earn from qualifying purchases.
| Reported measure | Raw kubectl | Radar MCP |
|---|---|---|
| Average tool calls per trial | 45.8 | 11.1; the original report called this 76% fewer calls |
| Average input tokens | 4.9 million | 2.3 million; reported as a 53% reduction |
| Average output tokens | 3,040 | 1,039; reported as a 66% reduction |
| Average agent time | 334 seconds | 169 seconds; reported as a 49% reduction |
| Pass rate | 77.6% | 80.8%, a 3.2 percentage-point increase |
| Diagnostic score | 0.765 | 0.862, a 0.097-point difference |
These are averages and scores reported by Dovzhikova for that 2026 setup; the tool-call, token, and time reductions are the report’s comparisons, not universal savings estimates. In particular, the original 76% call-reduction figure should not be treated as a settled headline: Radar’s later rerun says it did not replicate.
What the later rerun changed
Nadav Erell’s Radar / Skyhook post, dated July 20, 2026 and marked updated August 6, reports a separate run: 54 paired SREGym scenarios, Claude Sonnet 5 on both arms, and a three-node EKS cluster in us-east-1. The kubectl arm used raw commands through Bash, including exec; the Radar arm used Radar MCP tools with kubectl blocked. SREGym’s LLM judge, run at temperature zero, scored the first diagnosis submitted. This was not an exact reproduction of the earlier 52-scenario run. Read the updated rerun.
#1 Best Overall
| Reported measure | Raw kubectl | Radar MCP | Qualification |
|---|---|---|---|
| Pass rate | 87% (47 of 54 scenarios) | 91% (49 of 54 scenarios) | Small difference; Erell says he would not lean on the accuracy gap |
| Diagnostic score | 0.889 | 0.920 | Score reported for this rerun |
| Median time to correct diagnosis | 154 seconds | 41 seconds | Calculated only across the 44 faults both arms diagnosed correctly |
| Which arm was faster among those mutually correct cases | Faster in 1 case | Faster in 43 cases | Of the 44 faults both arms got right |
| Tool-call difference | Not 76% fewer in this rerun | Radar reports 43% fewer calls by mean and 19% by median | |
The revised post explains that the original timing included an attempted-fix stage even though the comparison was framed around diagnosis. It also notes that a raw call count treats calls with very different durations as equivalent. The later article therefore foregrounds time to a correct diagnosis and reports the call reduction by both mean and median. Erell says SREGym and its harness evolve, which makes exact reproduction difficult; the post offers replay of the newer scenarios.
What these results do—and do not—show
Both articles report a favorable result for Radar’s tool surface in their own setups, but neither isolates the MCP protocol as the cause. The compared arms differ in the information and tools presented to the agent: raw command output on one side, Radar’s structured cluster context on the other. Dovzhikova’s explanation is that agents using separate textual responses must repeatedly reconstruct resource ownership, service routing, and change sequences. Radar’s tools instead provide a resource graph and change timeline. Erell makes a similar distinction: a server that merely proxies kubectl would still return raw output. These are the publisher’s interpretations of its comparisons, not independently tested causal findings.
- The claim is about diagnosis in described fault-injection tasks. The reports do not establish better production remediation, uptime, safety, or performance across every Kubernetes workload.
- The comparisons have different scope. The original run used one model and 52 scenarios; the rerun used a different model and 54 paired scenarios. Their reported values should be read as separate experiments, not as one continuous measurement.
- The evidence is not independent replication. Both benchmark accounts come from Radar/Skyhook or an author who disclosed a Radar connection; the later account updates the earlier one but does not independently validate it.
- There is no industry-wide rate here. The reported figures belong to these benchmark configurations and should not be generalized to Kubernetes agents as a category.
How to assess a Kubernetes agent tool comparison
For teams evaluating another agent or MCP server, compare the information and control surface as well as the protocol. A fair test should define what counts as a correct diagnosis, when timing starts and stops, whether attempted fixes count, and how different-duration tool calls are measured. It should also disclose model, scenario set, cluster setup, permissions, and the artifact being graded.
- Diagnostic correctness: Does the evaluation require the actual root cause, or accept a plausible symptom description?
- Time and tool cost: Is the endpoint a correct diagnosis, and are both elapsed time and call duration considered? Token use and associated cost matter too.
- Context coverage: Can the tool surface connect resources, routing, ownership, and recent changes, or does the agent have to infer those relationships from separate outputs?
- Coverage and reproducibility: Are there multiple models and varied fault types? Can another team replay scenarios against a documented setup, accounting for changes to the harness?
- Operational controls: What can the agent read or change, how are secrets handled, and which write actions require additional controls?
Permissions and product claims
Dovzhikova’s original post describes Radar as respecting kubeconfig RBAC and as a single Go binary. Erell’s later post describes read-only tools, secret redaction, RBAC-enforced writes, and gated actions. These are product descriptions from Radar, not independent security certification. A team considering an agent for a real cluster should verify the permissions, secret handling, and write-action controls in its own environment rather than infer safety from a diagnosis benchmark.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
