Foundry IQ is not a new model and not a search box. It is Microsoft’s managed knowledge layer for enterprise agents: you group data sources and retrieval settings into a reusable knowledge base, Azure AI Search does the indexing and agentic retrieval underneath, and an agent consumes the result as a tool call that returns grounded, citable content.
Microsoft frames the problem in its own words: “How do I give an agent access to organizational knowledge and structured business data without building a custom connector for every system?” This article explains what Foundry IQ manages for you, what actually happens when an agent calls it, and which dependencies (identity, permissions, latency, cost, preview features) stay your responsibility.
As an Amazon Associate I earn from qualifying purchases.
What Foundry IQ is, and what it is not
Microsoft describes Foundry IQ as a managed knowledge layer for enterprise data. Three pieces are worth keeping separate, because the product names blur them:
Free tools Windows power users keep installed
One-click scans. No signup required.
| Piece | Role |
|---|---|
| Knowledge base | The reusable object. It combines one or more knowledge sources with settings that shape retrieval. Multiple agents can use the same one. |
| Azure AI Search | The required infrastructure underneath: indexing, indexers, and the retrieval engine. |
| Agentic retrieval | The name of the multi-query retrieval engine itself. Foundry IQ is the managed knowledge-base experience and set of integrations built around it. |
So “How is Foundry IQ different from Azure AI Search agentic retrieval?” has a short answer: agentic retrieval is the engine; Foundry IQ is the managed, reusable knowledge-base layer and integrations on top of it. Microsoft’s FAQ puts the value this way: “One Foundry IQ knowledge base provides access to multiple sources, removing the need to connect each agent to each source individually.”
It is also not a consumer feature that automatically reaches your company’s data. It is a developer workflow that needs an Azure subscription, a configured search service, and deliberate source connections.
How retrieval works
A calling application sends a query to the knowledge base, optionally with conversation history. What happens next depends on the configured reasoning effort:
- Query planning (when enabled). At low or medium effort, an LLM can break the request into focused subqueries, using conversation context and correcting things like spelling errors. At minimal effort, LLM query planning is skipped and retrieval is issued directly.
- Parallel search. Subqueries run in parallel against the configured knowledge sources.
- Semantic reranking. Results are semantically reranked.
- Aggregation. Results are combined into grounding content. Depending on configuration, the response also carries source references and an activity log showing what was done.
- Answer generation. The application or agent uses that grounding content to write the answer.
This design targets multi-part questions, follow-ups that depend on earlier turns, typo-ridden input, and queries that benefit from reformulation. The cost is time. Microsoft’s Azure AI Search overview says it plainly: “Agentic retrieval adds latency compared to a single-query pipeline, but it handles query complexity that a single query can’t.”
A caveat that matters for evaluation: better retrieval does not guarantee a correct answer. The extra steps improve what the model is given; the final response still needs to be grounded in it and tested against your own questions.
What “an agent tool call” means in practice
The clearest documented example is Microsoft’s hosted-agent quickstart. The agent never touches your indexes directly; it sees retrieval as one tool among others:
- Provision the knowledge base on your Azure AI Search service.
- Connect a toolbox to the knowledge base’s MCP endpoint.
- Deploy a hosted agent that discovers and calls the
knowledge_base_retrievetool. - At run time, the agent decides when a question needs organizational knowledge, invokes the tool, and receives cited source material to ground its answer.
The sample authenticates with managed identity, so no keys are embedded in the agent. Prerequisites include an Azure subscription, a configured Azure AI Search service, a Foundry project with model deployments, the necessary role assignments, and the managed identity setup. Treat it as infrastructure work, not a toggle.
That is the sense in which RAG becomes a tool call: instead of hard-coding “search, stuff results into the prompt” into each application, retrieval becomes a discoverable capability with a defined interface that any compatible agent can invoke when relevant.
Do you have to use Foundry Agent Service?
No. Azure AI Search is the required component; Foundry Agent Service is optional. Microsoft documents several ways to reach a knowledge base:
| Integration path | Notes |
|---|---|
| Foundry Agent Service (MCP toolbox) | The hosted-agent quickstart route described above; agent discovers knowledge_base_retrieve. |
| Microsoft Agent Framework | Agents built with the framework can call knowledge bases without running in Foundry Agent Service. |
| Custom application via REST API or supported SDK | Any app that supports the Azure AI Search knowledge-base APIs can query directly. |
| MCP-compatible host | The MCP tool route exposes the knowledge base to hosts that speak MCP. |
Foundry Agent Service itself does not charge for agent instances, per Microsoft’s FAQ; costs sit in the underlying services (see below).
Rank #4
Sources, indexing, and freshness
A knowledge base can mix indexed and remote sources, and they do not behave identically.
| Type | Examples named by Microsoft | Freshness |
|---|---|---|
| Indexed | Azure Blob Storage, OneLake, SharePoint, existing search indexes | Processed by Azure AI Search indexers; incremental refresh recurs on the schedule you configure, so data is only as fresh as that schedule. |
| Remote | For example, remote SharePoint via the Copilot Retrieval API | Queried at request time, so Microsoft says the data is current at query time. |
Don’t assume continuous refresh or uniform ingestion across sources. Choose per source: indexed for scale and ranking control, remote when staleness is unacceptable.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Preview status of sources
At Microsoft’s Build 2026 announcement, knowledge bases and selected sources were generally available. Additional sources, including Work IQ, Fabric IQ, File Search, Azure SQL, and MCP, were in preview at that time, and Web IQ via an MCP knowledge source was described as limited access. That announcement is the most recent status Microsoft has published here, and these labels change, so confirm the exact source, region, and status in current documentation before committing a design to a preview connector.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Security and identity
Microsoft documents three relevant mechanisms: ACL synchronization for supported indexed sources, permission enforcement at query time, and caller identity propagation through Microsoft Entra. Managed identity is the recommended way to connect Azure services to each other.
The limits are the important part:
- Document-level controls apply only where the knowledge source supports them and synchronization has been configured. Connecting a source does not by itself make each user’s permissions correct.
- Remote SharePoint goes through the Copilot Retrieval API and requires end users to hold a valid Microsoft 365 Copilot license.
- Permission behavior is therefore per source. Test with real user accounts that should, and should not, see specific documents before exposing an agent broadly.
Availability and cost
Availability and billing follow the underlying services: Azure AI Search and, where used, Azure OpenAI in Foundry Models. Azure AI Search has a free tier, and Microsoft describes a free token allocation for agentic retrieval. Beyond that allocation, agentic retrieval is billed by token consumption in Azure AI Search, while LLM query planning and answer synthesis can add separate Azure OpenAI charges. Rates vary by region and configuration, so price your actual setup rather than relying on a quoted figure.
The reasoning-effort setting is a cost lever as well as a latency one: minimal effort skips LLM planning, so it avoids that model usage entirely.
Microsoft’s performance claims
Microsoft’s Build 2026 Foundry blog reports two figures, both vendor-reported:
- Up to 20% improvement in answer quality in Microsoft’s benchmarks, across evaluated datasets, effort tiers, and model sizes.
- Up to 54% improved recall compared with single-shot RAG.
“Up to” means best cases. The blog does not say every workload will see these gains, and no independent or third-party benchmark was available for this article. Use them as a reason to run your own comparison, not as a forecast.
Foundry IQ versus a hand-built RAG pipeline
| Question | What to check |
|---|---|
| Source coverage | Is each connector you need generally available or still preview? |
| Permissions | Does each source support document-level authorization, and is it configured, with identity propagated from the caller? |
| Freshness | Scheduled indexer refresh or on-demand remote retrieval, source by source. |
| Quality vs. latency | Does query decomposition measurably help your questions enough to justify added latency? |
| Integration | Foundry Agent Service, Agent Framework, custom API/SDK, or an MCP host. |
| Total cost | Azure AI Search tokens after the free allocation, plus any Azure OpenAI usage for planning and synthesis. |
Foundry IQ pays off when several agents need the same sources and you would otherwise rebuild connectors, chunking, and retrieval logic for each. A simpler single-query pipeline can be the better choice for straightforward lookups over one corpus or when latency budgets are tight; at minimal reasoning effort, Foundry IQ itself approximates that behavior. Neither option is universally superior, so benchmark both on real questions from your users.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
