No. 21 of 29 ·AI Agent Evaluation Tools

VRUNAI

Web

Visit site Alternatives

Freedom report

Two barsScore 6.3

Free tierA free tier is on its own pricing page
Open codeNo open-source code on record
Runs widely1 of 6 device platforms
DocumentedPlans, terms and facts published

VRUNAI is ranked #21 of 29 in AI agent evaluation tools on Freedom251. It runs on Web. There is a free plan.

VRUNAI plans and pricing

All plans

Free Free Completely free; only LLM API calls to configured providers cost extra No VRUNAI fees, subscriptions, or usage limits · AGPL-3.0 vrunai.com · 7 Oct 2026

Compared on AI agent evaluation tools

Free plan: Yes
Evaluation methods: code
Tool-call checks: Yes
Trace ingestion: No
Safety evaluations: No

Best VRUNAI alternatives

See all 20

Two bars Future AGI AI Evaluation SDK Free tier No. 1$250/mo6.6 Two bars Promptfoo Free tier No. 2Free6.6 Two bars W&B Weave Free tier No. 3$60/mo6.6 Two bars DeepEval Free tier No. 4Free6.5 One bar Noveum Free tier No. 5$69/mo6.5 One bar Opik Free tier No. 6$19/mo6.5

The record

What it does: VRUNAI evaluates LLM agents across execution path, tool calls, and final outcomes.
Agent definitions: Users define tools, mock data, conditional flows, and test scenarios in YAML without writing code.
Consistency: It can run each scenario multiple times and score how consistently the agent follows the same path.
Cost tracking: It calculates cost per scenario and provider from token usage and includes model pricing and context window statistics.
Providers: The site lists OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, Ollama, and custom providers.
Interfaces: VRUNAI offers a terminal interface and a React web app, which can run locally or be tried in a hosted browser version.
Local model support: The FAQ says it supports local inference through Ollama and custom API endpoints, including OpenAI-compatible APIs.
Privacy: The CLI runs on the user's machine and the site says its keys stay in the terminal; web app keys are stored in browser localStorage and sent directly to configured provider APIs.
Backend and telemetry: The site states that VRUNAI has no accounts, backend, or telemetry and is fully client-side.
License: VRUNAI is open source under the AGPL-3.0 license.
Support: The site links to GitHub for bug reports and feature requests.
Notable cost limit: VRUNAI itself has no usage limits, but users pay for LLM API calls to the providers they configure.
Intended users: The site presents VRUNAI for people evaluating LLM agents and comparing their behavior across providers.
Price from: Free

Where it ranks on Freedom251

Best AI Agent Evaluation Tools in 2026#21 of 29

Is VRUNAI yours?

Claim it for free: prove the domain, then correct facts, plans and screenshots. An editor reviews every change.

Claim it free Promote VRUNAI

Sources

vrunai.com· checked 7 Oct 2026