Free and open first
Best AI LLM Evaluation Tools in 2026 — page 2
- 30 apps
- Research read 23 Sept 2026
- How the order is worked out
- 26
RAGChecker
Free plan Self-hostedChoose RAGChecker if you want an open-source, self-hosted way to measure RAG pipeline, retrieval, and generation performance.
- 27
Pydantic Evals
Free plan LinuxPydantic Evals provides a code-first way to check AI outputs and agent behavior, including against live traffic.
- 28
OpenAI Evals
Paid APISelf-hostedWebOpenAI Evals offers configurable dataset-based testing with several grader types and deployment choices.
- 29
UpTrain
Paid APISelf-hostedWebUpTrain combines predefined and custom LLM evaluations with failure analysis, experimentation, and regression controls.
- 30
Lighteval
Paid