DecodingTrust

Self-hosted · API

Visit site Alternatives

Freedom report

One barScore 5.3

Free tierNo free tier on record
Open codeNo open-source code on record
Runs widely0 of 6 device platforms
DocumentedPlans, terms and facts published

DecodingTrust is ranked #16 of 29 in LLM evaluation tools on Freedom251. It runs on API, Self-hosted.

Compared on LLM evaluation tools

Deployment options: self-hosted
Safety evaluations: Yes

Best DecodingTrust alternatives

See all 20

Two bars DeepEval Free tier No. 1Free6.6 Two bars Promptfoo Free tier No. 2Free6.6 One bar Giskard Free tier No. 3Free6.5 One bar Braintrust Free tier No. 4$249/mo6.4 One bar Confident AI Free tier No. 5$200/mo6.4 One bar Galileo Free tier No. 6$100/mo6.4

The record

Purpose: DecodingTrust is a research project for assessing trustworthiness in GPT models and helping researchers and practitioners understand LLM capabilities, limitations, and deployment risks.
Evaluation areas: The benchmark covers toxicity, stereotype and bias, adversarial robustness, out-of-distribution robustness, privacy, adversarial demonstrations, machine ethics, and fairness.
Models: The project says its evaluations mainly focus on GPT-3.5 and GPT-4, and it also supports causal LLMs hosted on Hugging Face or locally.
Resources: The project provides a dataset and evaluation scripts organized by trustworthiness area.
Reproducibility: The benchmark uses timestamped GPT-3.5 and GPT-4 model versions to support consistent results and reproducibility.
Installation: The project recommends cloning the repository and installing it in editable mode with pip so the data, code, and configurations remain together.
Supported architecture: The repository says it supports the ppc64le architecture on IBM Power-9 platforms.
License: The dataset and project are distributed under the CC BY-SA 4.0 license.
Content warning: The project warns that its data contains model outputs that may be considered offensive.
Model coverage limit: The repository says its benchmark mainly focuses on GPT-3.5-turbo-0301 and GPT-4-0314 for consistent conclusions and results.
Support: Questions and suggestions can be sent by GitHub issue or pull request, or by email to [email protected].
Intended users: The project describes its resources as intended to help researchers and practitioners assess LLM capabilities, limitations, and risks.

Where it ranks on Freedom251

Best LLM Evaluation Tools in 2026#16 of 29

Is DecodingTrust yours?

Claim it for free: prove the domain, then correct facts, plans and screenshots. An editor reviews every change.

Claim it free Promote DecodingTrust

Sources

decodingtrust.github.io· checked 4 Oct 2026
github.com/AI-secure/DecodingTrust· checked 4 Oct 2026