Production AI fails in ways a controlled demo will not reveal. Real users, stale or sensitive data, distributed permissions and operational pressure can produce incorrect outputs—or silently change records without triggering an error. The controls that matter are explicit permission boundaries, verifiable tool and data use, inspectable workflow state, and a named person or team accountable for the result.
That is the focus of InfoQ’s free, 60-minute online panel, AI in Production: What Breaks, What Works, and Who Approves It?, scheduled for October 14, 2026, alongside a Linux Foundation session on harness engineering for multi-agent systems.
As an Amazon Associate I earn from qualifying purchases.
What the production-AI panel covers
InfoQ’s agenda treats production readiness as more than model accuracy. It asks how much autonomy an agent should receive, how generated changes are verified, how sensitive data is protected, how retrieval-augmented generation (RAG) behaves with operational data, and how engineers justify stronger controls before an incident forces the issue.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →The five practitioners announced for the panel bring different operational perspectives:
#1 Best Overall
| Panellist | Production focus |
|---|---|
| Hien Luu | ML platforms, RAG, agents and reliability; author of MLOps with Ray |
| Katharine Jarmul | Privacy and security; author of Practical Data Privacy |
| Zichuan Xiong | Agentic systems and coding-agent harnesses |
| Premanand Chandrasekaran | Permissions, review gates and governance; co-author of Domain-Driven Design with Java |
| Lan Chu | Production data and ML pipelines, including RAG systems built on more than 10,000 documents |
The event’s central question is straightforward: can an organization prove what an AI system accessed, which actions it was permitted to take, how its output was checked and who remains accountable when it fails?
Why a successful prototype changes in production
A workflow that passes a controlled test can behave differently when exposed to real traffic. Users phrase requests unpredictably, records change after an index is built, permissions are spread across services, and latency or cost limits constrain what the system can do. Non-determinism also makes repeatable testing harder. A 2025 MLOps Community voice-agent panel described latency, accuracy, real-time processing, ambiguity and consistent user experience as recurring production problems; its participants noted that behavior changes substantially when usage grows from a prototype to hundreds or thousands of conversations.
RAG adds operational failure modes beyond the language model itself. A response can be fluent while relying on stale documents, poor retrieval, incomplete indexing or a source the user was not authorized to see. A production test therefore needs to evaluate freshness, retrieval behavior, access control, latency and failure handling—not only whether an answer sounds plausible.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Silent failures are more dangerous than visible errors
The Linux Foundation session Harness Engineering: From MAST’s Failure Taxonomy to MCP-Native Multi-Agent Production reports multi-agent failure rates of 41–86% across seven frameworks. Those figures come from the session description in 2026, which attributes the MAST (Multi-Agent Systems Failure Taxonomy) analysis to UC Berkeley and NeurIPS 2025 and says it covers more than 1,600 execution traces. They should be treated as reported session figures and checked against the underlying MAST publication before being used as independent research statistics.
MAST groups failures into system-design problems, inter-agent misalignment and task-verification failures. This classification matters because a system can complete a chain of calls without raising an exception and still fail the user’s objective.
Two incidents illustrate the gap
- A LangGraph customer-service agent made an unvalidated tool call that silently corrupted hundreds of Zendesk support tickets. The problem surfaced during an audit rather than through an application error.
- A Kubernetes-hosted presales assistant received an out-of-distribution input, entered an indefinite loop and consumed compute before operators detected it.
In both cases, “no error returned” was not evidence of a safe outcome. Operators needed records of tool calls, inputs, workflow state, resource use and the final effect on external systems.
What harness engineering adds
Harness engineering places enforceable controls around an agent instead of relying on a prompt or a model’s apparent good behavior. The Linux Foundation session describes a stack with four complementary layers:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
| Layer | Purpose in production |
|---|---|
| MCP-native tool contracts | Define which tools exist, what arguments they accept and what outputs they may return before an agent can invoke them. |
| LangGraph orchestration harnesses | Make multi-step workflow state and transitions explicit, so loops, retries and hand-offs can be inspected and bounded. |
| Pydantic schema contracts | Validate structured inputs and outputs at system boundaries rather than passing unchecked free-form data between components. |
| Opik telemetry | Record execution evidence for monitoring, task verification and post-incident diagnosis. |
These layers address different failure classes. Contracts constrain what an agent can request; orchestration controls how it proceeds; schemas reject malformed data; telemetry shows what actually happened. None replaces the others. A perfectly logged unsafe tool call is still unsafe, while a tightly scoped tool without telemetry may leave no usable explanation when a task goes wrong.
Decide autonomy by action, not by agent name
Governance works best when approval requirements are attached to concrete actions. The following policy pattern separates low-impact work from changes that affect money, records, access or regulated information:
| Action class | Typical control | Evidence to retain |
|---|---|---|
| Read-only retrieval from approved sources | Allow autonomously within the user’s permissions and a defined data scope. | Query, retrieved sources, identity and timestamp. |
| Drafting an answer, code change or customer reply | Permit generation, but require a person or automated policy check before publication. | Input context, generated artifact, checks and reviewer or policy decision. |
| Editing tickets, records or other external state | Require validated tool arguments, narrow field-level permissions and an approval gate for material changes. | Before-and-after values, tool call, authority and approval. |
| Deploying code, changing permissions or sending consequential communications | Use mandatory human approval, separation of duties and a reversible rollout where possible. | Test results, approver, deployment identity, scope and rollback status. |
| Accessing sensitive or regulated data | Allow only for an explicit purpose, with authorization checks, minimization and threat modeling before use. | Data classification, purpose, authorization decision and audit trail. |
This approach answers the panel’s governance questions directly: what may run autonomously, what requires a gate, which data may be retrieved and who owns the decision. The accountable owner should be named before deployment, not assigned after an incident.
Verification must cover the whole task
Checking the final text or code is insufficient when an agent can call tools or alter state. A production verification plan should test four boundaries:
Recommended Free Tools
- Generated changes: run tests, static analysis, security checks and policy rules before CI acceptance or deployment.
- Tool calls: validate arguments, authorization, rate limits, idempotency and the target resource before execution.
- Retrieved data: check source freshness, document-level permissions, provenance and whether retrieval actually supports the answer.
- Task completion: verify the intended external outcome, not merely that the model emitted a successful-looking message.
For high-impact actions, verification should be independent of the agent that proposed the action. A separate policy service, test suite or human reviewer is less likely to accept a plausible but self-confirmed result.
Best Value
Observability turns silent failure into a diagnosable event
Operators should be able to reconstruct the agent’s execution from durable records. At minimum, capture the user or service identity, model and prompt version, retrieved sources, tool names and arguments, sequence of calls, approvals, retries, resource consumption, outputs and changes made to external systems. Correlating those records with application and infrastructure logs helps distinguish a bad answer from a permission error, stale data, runaway loop or unintended mutation.
Telemetry also enables practical safeguards: maximum steps, timeouts, budgets for tokens or compute, loop detection, circuit breakers and automatic rollback for reversible changes. Alerting should cover abnormal tool volume, repeated retries, unusual data access and divergence between the requested task and the observed effect.
Trade-offs between common production approaches
| Approach | Autonomy and approval | Verification and auditability | Failure containment | Operating trade-off |
|---|---|---|---|---|
| Prompt-only agent | High apparent autonomy; approvals are implicit or absent. | Weak evidence about permissions, calls and completion. | Limited; malformed or unsafe actions can reach tools. | Low initial engineering effort, high incident uncertainty. |
| Tool-restricted agent | Autonomy limited to an allow-list and scoped credentials. | Better input validation, but still needs outcome checks and logs. | Improved blast-radius control. | More policy maintenance and integration work. |
| Harnessed multi-agent workflow | Explicit transitions, bounded retries and approval gates. | Contracts, schemas and telemetry make execution inspectable. | Strongest controls when limits and rollback are enforced. | Higher design, testing and observability cost. |
| Human-in-the-loop for consequential actions | Low autonomous authority at the final step. | Provides an accountable decision record, but reviewer quality matters. | Can stop harmful changes before execution. | Slower throughput and recurring review workload. |
There is no universal control pattern. A read-only internal search assistant can tolerate more autonomy than an agent that edits customer records or deploys code. The right design matches approval friction and verification depth to potential harm.
Free tools Windows power users keep installed
One-click scans. No signup required.
A production-readiness checklist
- Define the actions the agent may perform and the actions that always require approval.
- Issue least-privilege credentials and scope retrieval to the user’s authorization and the task’s purpose.
- Put schemas and tool contracts at every system boundary.
- Bound steps, retries, time, token or compute use, and external side effects.
- Test stale data, ambiguous requests, out-of-distribution inputs, permission failures and partial outages.
- Record prompts, versions, sources, calls, approvals, outputs and state changes in an auditable trail.
- Verify the external outcome independently of the agent’s own success message.
- Assign an accountable owner and rehearse rollback and incident investigation.
As Tom Shapland said on the 2025 MLOps Community panel, “There’s still a lot of craftsmanship that goes into taking your initial MVP and bringing it to something that’s production ready.” For agents, that craftsmanship is the combination of constrained authority, evidence-producing verification and governance that remains clear when the system behaves unexpectedly.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
