Best LM Evaluation Harness Alternatives in 2026

Updated October 2026

20 apps from AI LLM evaluation tools ranked against LM Evaluation Harness on the same published basis.

1
Vellum Free plan AndroidextensioniOSMacself-hosted $30/mo6.7
LM Evaluation Harness vs Vellum
2
Weights & Biases Free plan apiiOSLinuxMacself-hosted $60/mo6.7
LM Evaluation Harness vs Weights & Biases
3
Evidently AI Free plan apiLinuxMacself-hostedWeb $80/mo6.6
LM Evaluation Harness vs Evidently AI
4
Promptfoo Free plan apiLinuxMacself-hostedWeb Free6.6
LM Evaluation Harness vs Promptfoo
5
DeepEval Free plan LinuxMacself-hostedWin Free6.5
LM Evaluation Harness vs DeepEval
6
Giskard Free plan apiLinuxself-hostedWeb Free6.5
LM Evaluation Harness vs Giskard
7
Opik Free plan apiLinuxself-hostedWeb $19/mo6.5
LM Evaluation Harness vs Opik
8
Rhesis AI Free plan apiLinuxself-hostedWeb Free6.5
LM Evaluation Harness vs Rhesis AI
9
Braintrust Free plan apiself-hostedWeb $249/mo6.4
LM Evaluation Harness vs Braintrust
10
Confident AI Free plan apiself-hostedWeb $200/mo6.4
LM Evaluation Harness vs Confident AI
11
Galileo Free plan apiself-hostedWeb $100/mo6.4
LM Evaluation Harness vs Galileo
12
HoneyHive Free plan apiself-hostedWeb Free6.4
LM Evaluation Harness vs HoneyHive
13
Langfuse Free plan apiself-hostedWeb $29/mo6.4
LM Evaluation Harness vs Langfuse
14
LangSmith Free plan apiself-hostedWeb $39/mo6.4
LM Evaluation Harness vs LangSmith
15
LangWatch Free plan apiLinuxself-hostedWeb €29/mo6.4
LM Evaluation Harness vs LangWatch
16
Maxim AI Free plan apiself-hostedWeb $29/mo6.4
LM Evaluation Harness vs Maxim AI
17
NVIDIA NeMo Evaluator Free plan apiLinuxself-hosted Free6.4
LM Evaluation Harness vs NVIDIA NeMo Evaluator
18
Arize Phoenix Free plan apiself-hostedWeb Free6.3
LM Evaluation Harness vs Arize Phoenix
20
RAGChecker Free plan self-hosted Free6.3
LM Evaluation Harness vs RAGChecker
21
Inspect AI Free plan apiextension Free6.2
LM Evaluation Harness vs Inspect AI

LM Evaluation Harness alternatives compared

#	App	Score	Free plan	From	Runs on
1	Vellum	6.7	Free plan	$30/mo	Android, Browser, iOS, Mac, self-hosted, Web, Windows
2	Weights & Biases	6.7	Free plan	$60/mo	API, iOS, Linux, Mac, self-hosted, Web, Windows
3	Evidently AI	6.6	Free plan	$80/mo	API, Linux, Mac, self-hosted, Web, Windows
4	Promptfoo	6.6	Free plan	Free	API, Linux, Mac, self-hosted, Web, Windows
5	DeepEval	6.5	Free plan	Free	Linux, Mac, self-hosted, Windows
6	Giskard	6.5	Free plan	Free	API, Linux, self-hosted, Web
7	Opik	6.5	Free plan	$19/mo	API, Linux, self-hosted, Web
8	Rhesis AI	6.5	Free plan	Free	API, Linux, self-hosted, Web
9	Braintrust	6.4	Free plan	$249/mo	API, self-hosted, Web
10	Confident AI	6.4	Free plan	$200/mo	API, self-hosted, Web
11	Galileo	6.4	Free plan	$100/mo	API, self-hosted, Web
12	HoneyHive	6.4	Free plan	Free	API, self-hosted, Web
13	Langfuse	6.4	Free plan	$29/mo	API, self-hosted, Web
14	LangSmith	6.4	Free plan	$39/mo	API, self-hosted, Web
15	LangWatch	6.4	Free plan	€29/mo	API, Linux, self-hosted, Web
16	Maxim AI	6.4	Free plan	$29/mo	API, self-hosted, Web
17	NVIDIA NeMo Evaluator	6.4	Free plan	Free	API, Linux, self-hosted
18	Arize Phoenix	6.3	Free plan	Free	API, self-hosted, Web
20	RAGChecker	6.3	Free plan	Free	self-hosted
21	Inspect AI	6.2	Free plan	Free	API, Browser

Make your app an alternative to LM Evaluation Harness

The sponsored alternative slot on this page is labelled Sponsored.

See the price

Questions about LM Evaluation Harness alternatives

What is the best alternative to LM Evaluation Harness?

Vellum, number 1 in AI LLM evaluation tools with a score of 6.7 out of 10. The others here: Weights & Biases, Evidently AI, Promptfoo and 16 more.

What is the best free alternative to LM Evaluation Harness?

Vellum is the best-ranked alternative with a free plan. 20 of the 20 alternatives here publish a free plan on their own pricing pages.

How are these alternatives ranked?

Ranked free-and-open first: a usable free tier and open-source code, then the platforms it runs on and its documentation.