ChainForge

Overview

ChainForge is an open-source visual environment for designing prompt experiments and evaluating language-model responses. It suits developers and researchers who want to compare outputs systematically across models and prompt variations. Its strongest case is a free local installation; the web version is more convenient to share, but has fewer features and requires API keys to be entered again each time it loads.

Key features

ChainForge can query several LLMs at once and vary prompts, models and settings, so users can examine how changes affect responses side by side. Code-based and LLM-based scorers evaluate outputs, and plots help surface differences across an experiment. Optimization is manual, making this a better fit for users who want to direct their own tests than for anyone looking for automated prompt improvement.

Chat Turn nodes support multi-turn experiments: users can continue several conversations, template messages and pass a conversation from one model to another. That makes ChainForge useful for probing conversational behavior, not only one-shot completions. It can import JSONL, XLSX and CSV spreadsheets, then export inspected response data as XLSX.

Supported providers include OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock and Ollama. Local installations can also connect custom providers written as Python completion functions. The web version can create a unique link for sharing an experiment, but it lacks some local capabilities, including Python evaluation code, environment-variable API keys and locally hosted Alpaca or Llama models through Dalai.

Pricing

Free and open-source: free. The plan has no stated limits, which makes ChainForge a practical option for individual experimentation without a subscription. It does not include API access, and optimization is manual. Users still need access to model providers and their credentials; the web version requires keys to be set again on each load.

The choice between web and local use matters more than a paid tier: the browser version has a limited feature set, while local installation supports environment-loaded keys, Python evaluators and local models. Locally, secure mode can encrypt settings or all saved files. Forgetting the password makes encrypted flows or settings inaccessible, and the password cannot be recovered.

Platforms

ChainForge is available as a web app or for self-hosted local installation. The beta installation guide supports Chrome, Firefox, Edge and Brave, not other browsers. Its documentation describes the project as an open beta in active development, so it is a better fit for users comfortable with a developing tool than for teams that require a mature, supported commercial product.

Who it's for

Choose ChainForge if you want a visual, provider-flexible workspace for comparing prompts, models and conversation flows, especially if Python evaluators or local models are part of your workflow. Look elsewhere if you need automated optimization, API access, a full-featured browser experience or a guaranteed support channel: questions and requests go through GitHub issues or a Google Form.

Pros and cons

• Pros: Parallel queries, configurable scorers and plotted results support structured comparisons across models and prompts.
• Pros: Local Python evaluators, custom providers and locally hosted models give technical users room to extend experiments.
• Pros: Spreadsheet and JSONL imports plus XLSX export connect experiments to existing data workflows.
• Cons: The web version has fewer features and requires users to re-enter API keys on each load.
• Cons: Manual optimization leaves prompt iteration to the user, and the product is still in open beta.
• Cons: Secure-mode encryption carries a serious recovery trade-off: a forgotten password means losing access to encrypted saved work.

Alternatives

If you want a managed prompt and evaluation platform with a free tier and paid options, consider Vellum. For a free project from Microsoft Research authors, PromptWizard is another option. Choose OpenPrompt if you want an open-source prompt-learning framework with stated Python and PyTorch requirements. Agenta offers a free Hobby plan with team, run, evaluation and retention limits, which may suit teams seeking a defined hosted allowance. Google Cloud Agent Evaluation is an alternative for users who prefer usage-based pricing for evaluation. Latitude has a free Starter plan with monthly credits, data retention and unlimited seats. For prompt optimization in a web app, Prompt Engine offers a limited free Hobby plan and a paid Pro option. PromptEval is another freemium option, with monthly web-evaluation and API-lint caps on its free plan.

Browse more tools in AI Prompt Generators or AI Prompt Management Software.

Verdict

ChainForge is a strong choice for technically minded users who want free, visual prompt and model comparisons with scoring and multi-turn testing. Its local version is the main reason to choose it; the limited web feature set, manual optimization and beta status are reasons to look elsewhere if you need a polished, automated or fully browser-based workflow.