Mycelium is an open-source project that tries to answer one narrow question for AI agent builders: given a natural-language request, which agent endpoint should handle it? Instead of asking a large language model to pick from a tool list on every call, it embeds the request and searches a local vector index. The project reports cold discovery in about 9.56 ms and 70.7% top-1 intent accuracy on a synthetic benchmark of 100,000 agents. Those are the project’s own figures. The sources available for this article include no independent reproduction, so treat them as claims to test against your own tools, not as measured properties of your deployment.
What Mycelium is
Mycelium describes itself as an open-source semantic registry and routing protocol for agentic workflows. Its GitHub README says it combines a local ChromaDB vector store, the all-MiniLM-L6-v2 sentence-embedding model, and a FastAPI service to resolve natural-language intent to agent endpoints. It publishes Python and JavaScript SDKs:
As an Amazon Associate I earn from qualifying purchases.
- Python:
pip install mycelium-agents - JavaScript:
npm install mycelium-js
The project frames its problem as “The Tool Routing Bottleneck” and its pitch as “sub-10ms semantic tool routing” with “No LLM overhead.” Those phrases are useful search terms, but they are the project’s own wording, not independent findings.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallHow a routing decision works, as the project describes it
The README’s design implies the following path. Treat it as the project’s description of its architecture rather than a verified trace of the code.
- The calling agent passes a natural-language description of what it needs.
- The description is embedded locally with
all-MiniLM-L6-v2, so no hosted model is called to choose a tool. - ChromaDB returns the closest registered agent entries from the local index.
- The matched endpoint is handed back to the caller, which then invokes it.
The step that matters most for your evaluation is step 3. Routing quality depends entirely on how well the registered descriptions separate one agent from another, and an embedding model compares meaning at the level of similarity, not at the level of what an action will do.
The numbers the project reports
Every figure below is self-reported by Mycelium. The table keeps each number tied to the source that published it, because the two main sources do not describe the same chain test.
| Claim | Mycelium result | Comparison given in the source | Where published |
|---|---|---|---|
| Top-1 intent accuracy | 70.7% | BM25: 40.4% (a 30.3 percentage-point gain) | Project announcement, September 27, 2026 |
| Cold discovery latency | 9.56 ms | BM25: 194.0 ms | Project announcement, September 27, 2026 |
| End-to-end, two-hop weather-to-translation chain | 37.6 ms | Not stated in the announcement | Project announcement, September 27, 2026 |
| Throughput and errors, 100 concurrent workers | 130+ requests per second; 0.0% errors | Not stated in the announcement | Project announcement, September 27, 2026 |
| Native three-hop chain | 36.25 ms | Not stated in the README | GitHub README, performance table labeled v0.3.0 |
| P95 latency | 11.4 ms | Not stated in the README | GitHub README, performance table labeled v0.3.0 |
| Single-node throughput | Above 130 requests per second | Not stated in the README | GitHub README, performance table labeled v0.3.0 |
The two chain figures should not be merged. The announcement measures a two-hop chain at 37.6 ms, while the README reports a three-hop native chain at 36.25 ms. A longer chain at a lower number is not evidence that the system got faster; it reflects different tests.
Conditions behind the figures
A sub-10 ms headline is easy to misread, so the conditions matter as much as the number.
- Corpus: The announcement evaluates a synthetic corpus of 100,000 agents and 441 task-oriented queries. A synthetic catalog does not reproduce the overlapping, inconsistently written tool descriptions found in a real deployment.
- Cache state: The 9.56 ms figure is cold discovery, meaning the cache was cold for the measurement. It is not a steady-state average over a warm service.
- Embedding time: Query embedding is included in the timing. That is the honest choice, and it is the right baseline for comparing against any other router.
- Hardware: The announcement describes commodity CPU hardware without naming a specific processor, core count, or memory configuration, so the figures cannot be scaled to your servers.
- Scope: The sub-10 ms claim covers discovery only. End-to-end chains in both sources take roughly 36 to 38 ms, which is the number a user waiting on an agent workflow actually experiences.
- Concurrency: The 130+ requests per second figure in the announcement is tied to 100 concurrent workers. The README gives a single-node throughput figure without the same test description, so do not assume the two are the same test.
What is not established
- Independent replication: The sources available for this article include no third-party benchmark or reproduction of any Mycelium figure.
- Head-to-head comparison: No controlled test compares Mycelium with another production router.
- Methodological detail: The published material does not give enough detail about how the benchmark queries were written or labeled to infer how the router will perform on production traffic.
- Error type: The headline accuracy does not separate a wrong route from a request that was ambiguous, and the sources do not describe a fallback policy for requests that match no agent well.
Independent context: a different approach to the same problem
The 2026 ACL Industry Track paper LatentGate: Low-Latency Semantic Routing via Frozen-Backbone Probing of Small Language Models, by Shivam Ratnakar, Abhiroop Talasila, and Vinayak K Doifode, is the most relevant independent work on the same problem. It takes a different route: it probes a frozen small language model rather than doing a nearest-neighbor lookup over embeddings. The paper also warns that embedding-based routers can collapse semantically similar but functionally distinct agents, which is exactly the failure a catalog with near-duplicate tools would expose.
| Aspect | Mycelium (project-reported) | LatentGate (ACL 2026 Industry Track) |
|---|---|---|
| Method | Local embeddings with a ChromaDB vector index | Frozen-backbone probing of small language models |
| Accuracy | 70.7% top-1 on a synthetic 100,000-agent corpus | 98.8% in-domain and 80.0% out-of-domain on natural queries across 100 enterprise agents |
| Latency | 9.56 ms cold discovery, CPU, query embedding included | About 28 ms runtime on a T4 GPU |
| Evaluation status | Project announcement and README, not independently reproduced | Peer-reviewed conference paper |
The two rows are not comparable. The datasets, query sets, accuracy definitions, and hardware all differ, so the table shows the shape of the trade-off rather than a ranking. LatentGate does not validate Mycelium, and Mycelium’s numbers do not show that LatentGate is slower in practice. A vendor-authored engineering article from StackOne, dated May 12, 2026, describes semantic discovery over SaaS connector actions. It is a related category worth including in your shortlist, but it is a vendor’s description, not an independent evaluation.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Safety controls and their limits
The project announcement says Mycelium includes a bridge for Anthropic’s Model Context Protocol and a Human-On-The-Loop guard. Under that design, read-only intents may execute automatically, while mutating intents are intercepted and held until a human provides cryptographic authorization. The material available for this article does not explain how that authorization is implemented.
Recommended Free Tools
This is a control design the project describes. No third-party security audit, threat model, formal verification, or independent penetration test appears in the sources available for this article. Treat the guard as a design intent to verify, not as a guarantee that the system is secure.
How to evaluate a fast semantic router before you rely on it
Use this sequence to test Mycelium, or any semantic router, against your own tools and risk level.
- Build a labeled set from real traffic. Map real requests to the correct endpoint, and include requests that should be refused or sent to a human. Do not tune agent descriptions against the same examples you later score.
- Score held-out requests on two measures. Record top-1 accuracy and wrong-route rate separately, because a router can be mostly right and still send a costly request to the wrong tool.
- Time the full path on your hardware. Include query embedding, the lookup, and the call that follows. Record cold and warm cache separately, and report P95 and P99 rather than averages.
- Add near-duplicate tools. Register tools with similar descriptions but different side effects, such as “update customer record” and “read customer record.” Re-run step 2 to test the collapse risk the LatentGate paper describes.
- Test the mutating path. With the approval guard enabled, confirm that a mutating intent is held, that nothing executes before approval, and that you can log and reverse the action.
Matching the test to the stakes
- Read-only tools: Accuracy and tail latency are the main risks. A wrong read is usually recoverable.
- Tools that write data: Wrong-route rate on near-duplicate tools matters more than milliseconds, and the approval path must be tested end to end.
- Tools that move money or change access: Do not rely on the router’s own guard alone. Put an independent authorization and audit layer in front of the action.
The Bottom Line
Mycelium is a reasonable candidate for a local first-stage router that avoids an LLM call on every tool choice, and its low latency figures are worth testing. Its headline numbers, though, come from a synthetic benchmark run by the project itself, and the sub-10 ms figure measures discovery, not the full workflow. Adopt it only after it passes a test on your own labeled requests, your near-duplicate tools, and your approval path.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
