PromptMetrics

Free plan APILinuxMacSelf-hostedWebWindows

What PromptMetrics is

PromptMetrics is a free, open-source, self-hosted registry for managing prompts and observing LLM activity. Prompts can be stored, versioned, retrieved, and rolled back through a REST API or CLI, with changes recorded in Git. Structured logs capture model, token, latency, cost, and custom-tag metadata, with export to stdout JSON or OpenTelemetry. The project also tracks agent traces, spans, workflow runs, and prompt versions labeled by environment. Evaluation suites can run against datasets with budget tracking, while A/B tests compare prompt versions and can promote a winner. Its compliance scanner checks for PII, API keys, URLs, and IP addresses and assigns risk scores. The playground can proxy calls for OpenAI, Anthropic, Cohere, Ollama, and Azure OpenAI. Prompt content can use filesystem, GitHub, or S3 storage; metadata can use SQLite or PostgreSQL. Interfaces include an optional Next.js dashboard, CLI, REST API, and Node.js and Python SDKs. Installation options are npm, Docker Compose, or building from source.

Who it suits

PromptMetrics may suit teams that want to self-host prompt versioning, LLM activity logs, evaluations, and compliance checks. It is aimed at users who can deploy and operate a self-hosted project through its documented installation options.

What it does well

Tracks prompt changes in Git and supports rollback.
Exports logs through stdout JSON or OpenTelemetry.
Supports dataset evaluations and A/B comparisons.
Scans prompts for sensitive patterns and assigns risk scores.
Offers Node.js and Python SDKs.

What to know first

The project is self-hosted rather than a hosted service.
The README sets a default limit of 300 requests per 60-second window per API key.
Security guidance recommends HTTPS, a reverse proxy, and regular backups.

Limits

The README lists a default limit of 300 requests per 60-second window for each API key; environment variables can change that limit.

Getting started

Open the PromptMetrics GitHub website
Install it with npm, Docker Compose, or a source build
Choose filesystem, GitHub, or S3 for prompt content
Configure SQLite or PostgreSQL for metadata
Use the REST API, CLI, SDKs, or optional Next.js dashboard

Verdict

PromptMetrics combines prompt management with logging, evaluations, agent traces, and compliance scanning. Its self-hosted deployment and operational security guidance are central considerations.

Questions

Is PromptMetrics free?

Yes. It is free and open source, and the repository identifies it as MIT licensed.

Which model providers can its playground proxy?

It can proxy chat and completion calls for OpenAI, Anthropic, Cohere, Ollama, and Azure OpenAI.

What interfaces and SDKs are available?

PromptMetrics provides a REST API, CLI, optional Next.js dashboard, and Node.js and Python SDKs.

Where can prompt content and metadata be stored?

Prompt content can use filesystem, GitHub, or S3 storage. Metadata can use SQLite or PostgreSQL.

How is the dashboard protected against key exposure?

The dashboard authenticates through a proxy, so API keys are not stored in browser localStorage.

Facts from PromptMetrics's own pages

Agent telemetry: It tracks agent traces and spans, workflow runs, and prompt versions tagged with environment labels. · github.com, 8 Oct 2026
Compliance scanning: Its compliance engine scans prompts for PII, API keys, URLs, and IP addresses and assigns risk scores. · github.com, 8 Oct 2026
Dashboard: An optional Next.js dashboard provides charts and views for prompts, logs, traces, evaluations, A/B tests, compliance, and the playground. · github.com, 8 Oct 2026
Dashboard key handling: The dashboard authenticates through a BFF proxy so API keys are not stored in browser localStorage. · github.com, 8 Oct 2026
Deployment: The README provides installation options using npm, Docker Compose, or source, and describes the product as self-hosted. · github.com, 8 Oct 2026
Evaluations and A/B tests: Evaluation suites can be run against datasets with budget tracking, and A/B tests compare prompt versions and can promote a winner. · github.com, 8 Oct 2026
Evaluations and testing: It supports evaluation suites, dataset-backed evaluation runs, and statistical A/B comparisons with winner promotion. · github.com, 8 Oct 2026
Headquarters: Berlin, Germany · github.com, 28 Sept 2026
Installation: The README documents installation through npm, Docker Compose, or building from source. · github.com, 8 Oct 2026
Interfaces and SDKs: The project provides a REST API, CLI, optional Next.js dashboard, and Node.js and Python SDKs. · github.com, 8 Oct 2026
License: The repository identifies the project as MIT licensed. · github.com, 8 Oct 2026
LLM providers: The playground can proxy chat and completion calls for OpenAI, Anthropic, Cohere, Ollama, and Azure OpenAI. · github.com, 8 Oct 2026
Logging: Structured logs can capture model, token, latency, cost, and custom-tag metadata and export to stdout JSON or OpenTelemetry. · github.com, 8 Oct 2026
Maker and company details: The repository says it is maintained by the PromptMetrics team; it does not state a headquarters or founding date. · github.com, 8 Oct 2026
Observability: It logs LLM request metadata such as model, tokens, latency, cost, and custom tags, and can export data through OpenTelemetry. · github.com, 8 Oct 2026
Prompt versioning: Prompts can be stored, versioned, retrieved, and rolled back through a REST API or CLI, with changes recorded in Git. · github.com, 8 Oct 2026
Purpose: PromptMetrics is a lightweight, self-hosted prompt registry with Git-backed versioning, metadata logging, and evaluations for LLM observability. · github.com, 8 Oct 2026
Rate limit: The README lists a default limit of 300 requests per 60-second window per API key, configurable through environment variables. · github.com, 8 Oct 2026
SDKs: The project lists Node.js and Python SDKs for programmatic access. · github.com, 8 Oct 2026
Security controls: API keys are HMAC-SHA256 hashed, support scoped permissions and optional expiration, and the dashboard uses a proxy so keys are not stored in browser localStorage. · github.com, 8 Oct 2026
Security guidance: The security policy recommends HTTPS, a reverse proxy, a long random API key salt, and regular database backups. · github.com, 8 Oct 2026
Storage: Prompt content is stored in Git, while metadata can use SQLite or PostgreSQL; filesystem, GitHub, and S3 storage drivers are listed. · github.com, 8 Oct 2026
Storage options: Prompt content can use filesystem, GitHub, or S3 storage, while metadata can use SQLite or PostgreSQL. · github.com, 8 Oct 2026
What it does: PromptMetrics is a lightweight, self-hosted prompt registry for versioning prompts and tracking LLM activity. · github.com, 8 Oct 2026

Where PromptMetrics stands on Freedom251

Prompt Management Software#17 of 26
AI Prompt Management Software#18 of 25