Freedom251 › AI LLM Evaluation Tools
Rhesis AI
Plans
| Plan | Billed and limits | Price |
|---|---|---|
| Self-hosted open source edition | Nothing metered. | Free |
| Rhesis Cloud Free | Start free on Rhesis Cloud with a monthly allowance; exact allowance not stated on the opened pricing page. | Free |
As rhesis.ai lists them, read 30 Sept 2026. The maker's own page has the final word.
Facts from Rhesis AI's own pages
- API and CI/CD
- The REST API supports managing test sets, triggering test runs, fetching results, and inspecting traces for custom integrations and CI/CD pipelines. · docs.rhesis.ai, 30 Sept 2026
- Deployment and data location
- Rhesis can run as a managed service, locally with Docker, or self-hosted; the site says it is developed in Germany and hosted in the EU. · docs.rhesis.ai, 30 Sept 2026
- Evaluation integrations
- Rhesis supports DeepEval and Ragas metrics and can import Garak probes as test sets for adversarial scanning. · docs.rhesis.ai, 30 Sept 2026
- Evaluations
- Rhesis runs test sets against an application endpoint, evaluates responses using configured metrics, and records results in test runs. · docs.rhesis.ai, 30 Sept 2026
- Headquarters
- Potsdam, Germany · rhesis.ai, 28 Sept 2026
- Intended users
- Rhesis describes itself as built for experts, engineers, and product managers working on AI agents. · rhesis.ai, 30 Sept 2026
- Model integrations
- Rhesis supports cloud and local model providers through LiteLLM, including OpenAI, Anthropic, Google Gemini, Mistral, Cohere, Groq, Together AI, Ollama, and vLLM. · docs.rhesis.ai, 30 Sept 2026
- Product
- Rhesis is an open-source collaboration layer that brings domain-expert knowledge into AI agent development. · rhesis.ai, 30 Sept 2026
- Security testing
- Rhesis integrates Garak to probe applications for issues such as prompt injection, data leakage, and harmful content generation. · docs.rhesis.ai, 30 Sept 2026
- Support
- The getting-started page points users to in-app support, documentation, a support email, GitHub, and Discord. · docs.rhesis.ai, 30 Sept 2026
- Team workflow
- Domain experts, engineers, and product managers can share a workspace for reviews, annotations, and expectations attached to test cases and agent versions. · rhesis.ai, 30 Sept 2026
- Test generation
- Rhesis can generate test sets from application requirements or constraints, including single-turn prompts and multi-turn conversations. · docs.rhesis.ai, 30 Sept 2026
- Tool integrations
- The integrations page lists Notion, GitHub, Jira, GitLab, Asana, Azure DevOps, Linear, and Shortcut as tool connections. · docs.rhesis.ai, 30 Sept 2026
- Tracing
- Rhesis Tracing uses OpenTelemetry to capture LLM calls, tool invocations, and retrieval operations in test or production mode. · docs.rhesis.ai, 30 Sept 2026
Where Rhesis AI stands on Freedom251
- AI LLM Evaluation Tools#22 of 30