Inferwall

What Inferwall is

Inferwall is self-hosted software that checks inputs and outputs in LLM applications for threats such as prompt injection, jailbreaks, data leakage, and unsafe content. Its detection combines heuristic, machine learning, semantic, and configurable LLM-judge layers. It can run as a Python SDK, an HTTP server, or a transparent reverse proxy, with listed wrappers for OpenAI, Anthropic, LangChain, and FastAPI. Other supported integrations include Ollama, Hugging Face, LlamaIndex, Pydantic, vLLM, and Mistral. Teams can set allow, flag, or block thresholds by route, and add or override signatures and version policies under ~/.inferwall/. The Lite profile is about 8 MB and needs no GPU; Standard includes about 730 MB of models, while Full adds the LLM-judge layer and requires LLM credentials. The maker says its Rust heuristic engine runs in under 0.3 ms at p99. The engine uses Apache 2.0, and community signatures use CC BY-SA 4.0. Inferwall cautions that detection is imperfect and should be one part of a broader defense strategy. It is free and open source.

Who it suits

Inferwall may suit teams operating LLM applications that want configurable checks for inputs and outputs and control over deployment. Its self-hosted options and listed integrations can fit teams using the supported frameworks and model stacks.

Limits

The Lite profile is about 8 MB and requires no GPU. Standard includes about 730 MB of models; Full adds the LLM judge layer and requires LLM credentials. The project cautions that detection is imperfect.

Getting started

1. Visit https://www.inferwall.com/.
2. Choose a deployment option: Python SDK, standalone HTTP server, or transparent reverse proxy.
3. Select a profile: Lite is about 8 MB, Standard includes about 730 MB of models, and Full adds the LLM judge layer.
4. Connect a listed wrapper or supported stack integration.
5. Set route thresholds and add or override signatures under ~/.inferwall/.

Verdict

Inferwall offers several deployment options, configurable detection layers, and controls for tailoring signatures and thresholds. Its own caveat is important: it is one layer in a defense-in-depth approach, not a guarantee against threats.