Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Grafana’s approach treats AI-agent quality as a set of measurable service-level objectives (SLOs): evaluate observable behavior, convert the result into a pass rate, and use the tolerated miss rate as an error budget. The method gives teams a controlled way to ship prompt and model changes without hiding failures inside a single quality score.
Why ordinary uptime metrics miss agent quality
An agent can respond quickly, produce no infrastructure errors, and still give an unsupported answer, fail to complete the user’s task, expose private information, or follow a malicious instruction. Latency, availability, exceptions, and token counts describe how the system ran; they do not prove that its behavior was correct.
As an Amazon Associate I earn from qualifying purchases.
Grafana Labs’ September 24, 2026 article proposes adding behavioral evaluations to the same reliability discipline used for other production services. The evaluation result becomes a rate or percentage, that rate receives a target, and the gap between the target and actual performance becomes a budget for tolerated failures.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsHow an agent behavior becomes an SLO
- Define an observable behavior. Choose a question that can be judged from a conversation, message, tool call, or trace, such as whether an answer is grounded in supplied documents or whether a requested action was fulfilled.
- Run an evaluation. A check classifies each relevant event as a pass or failure, producing a score or pass percentage.
- Set a target. Give that behavior its own reliability objective rather than combining unrelated properties.
- Calculate the error budget. The allowed miss rate is the portion of evaluations that may fail during the SLO window.
- Use the remaining budget to govern change. Teams can spend budget on experiments while performance is healthy and slow or pause changes when failures consume it too quickly.
Measure separate failure modes instead of one blended score
Grafana explicitly warns against collapsing quality into a single composite number. An agent may be excellent on one dimension and unacceptable on another, so each important behavior needs a separate target and alerting policy.
#1 Best Overall
| Behavior or operational dimension | What an evaluation or metric can test | Why it deserves its own objective |
|---|---|---|
| Groundedness | Whether claims are supported by the available context or sources | A response can be fluent and fast while still inventing facts |
| Fulfillment | Whether the conversation actually completes the user’s requested outcome | Correct-sounding text is not the same as completing a task |
| Toxicity | Whether generated content violates the team’s safety standard | A small safety failure may matter even when other quality scores are high |
| Privacy leakage | Whether protected or sensitive information appears in output or tool activity | Privacy incidents should not be averaged away by good responses elsewhere |
| Prompt-injection resistance | Whether the agent ignores hostile instructions embedded in data or tools | Security behavior can regress independently of helpfulness |
| Latency and cost | p95 response latency and estimated daily spend | Operational limits affect user experience and budgets even when answers are correct |
Deterministic checks and model-based judges
Deterministic evaluators
Rule-based checks are appropriate when the failure has a clear signature. Grafana gives a leaked API key as an example: a pattern matcher or secret detector can flag it consistently and cheaply. Similar checks can enforce required fields, forbidden strings, or tool-call constraints when the rule is unambiguous.
Model-based evaluators
A model judge can assess qualities that require interpretation, including groundedness, fulfillment, toxicity, privacy leakage, and resistance to prompt injection. Because a judge is itself probabilistic, teams should inspect representative conversations and keep its rubric stable enough to make trend changes meaningful. Grafana Cloud Agent Observability supports both deterministic and model-based evaluations and guards.
Rank #2
The error-budget example and the two time windows
Illustrative target
Grafana’s article uses 95% fulfilled conversations over 30 days as a conceptual example. That target leaves a 5% error budget: up to 5% of evaluated conversations may miss the fulfillment criterion during the example period. The figure illustrates the method; it is not a published benchmark or universal target for every agent.
Documented product implementation
In the product workflow, an online evaluation provisions an SLO in Grafana’s SLO application over a rolling 28-day window and attaches the relevant evaluation labels. The 30-day period belongs to the article’s teaching example, while 28 days is the documented implementation detail for provisioned online-evaluation SLOs.
How teams use the budget to manage changes
When budget remains
A healthy balance gives a team permission to try a new prompt, model, tool, or feature while accepting the planned amount of failure. The budget makes that risk explicit instead of treating every experiment as an emergency.
When the budget burns down
Rapid consumption is a signal to slow releases and investigate. Grafana recommends replaying known conversations before shipping prompt or model changes, checking that the proposed version stays inside each behavioral target. This catches regressions that infrastructure-only tests cannot see.
Why a single bad conversation is not enough
The practical alerting question is burn rate and trend. One poor answer may be an isolated miss; a gradual decline can quietly consume the budget. Tracking the rate over time reveals accumulation and supports a measured response.
What Grafana Agent Observability connects
Agent Observability is designed to follow a request from the user-facing entry point through planning, tool invocations, large-language-model calls, and the final response. Within that trace, Grafana says teams can correlate:
- Agent and version identifiers
- Prompt changes and tool usage
- Evaluation and quality scores
- Latency and runtime errors
- Token consumption and estimated costs
The workflow includes online evaluations and guards for production traffic, offline experiments for testing changes, and conversation inspection for reviewing individual outcomes. Grafana lists integrations for LangChain, LangGraph, OpenAI Agents, LlamaIndex, Google ADK, and the Vercel AI SDK.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.A practical rollout sequence
- Instrument the complete trace. Capture the user request, planning steps, tool calls, model calls, final answer, latency, errors, and token usage so a failed evaluation has enough context for diagnosis.
- Start with a small set of behaviors. Pick the failures that create the greatest user, safety, or business risk. Keep groundedness, fulfillment, toxicity, and security-related checks distinct.
- Match each behavior to an evaluator. Use deterministic rules for unmistakable patterns and model-based judges for semantic criteria. Record the evaluator version and rubric with the result.
- Run offline experiments. Replay a representative set of known conversations against the current and proposed prompt or model. Inspect borderline cases rather than relying only on the aggregate score.
- Create online evaluations. In Grafana Cloud, provision an SLO from the online evaluation, use its evaluation labels, and set the target and rolling 28-day window appropriate to that behavior.
- Define burn-rate responses. Decide who investigates, which changes are paused, and what evidence is required before the budget is spent further.
- Review trends with operations data. Put quality, p95 latency, errors, token use, and estimated daily cost on the same dashboards so a quality regression can be weighed against its operational cause.
Alerting beyond quality scores
Grafana describes dashboards that can alert on p95 latency against SLO targets and on estimated daily cost. Those signals complement behavioral SLOs: an agent may remain grounded and fulfill requests while becoming too slow or expensive, or it may stay fast while quality drifts downward. Burn-rate views help distinguish a transient miss from a sustained trend that threatens the objective.
How usage is billed
Grafana describes usage billing for Agent Observability across three dimensions:
Recommended Free Tools
| Billing dimension | What it measures |
|---|---|
| Generations | Generations produced by monitored agents |
| Evaluation and guard tokens | Tokens consumed by large-language-model-based evaluations and guards |
| Standard telemetry | Telemetry emitted by the software development kits |
The official material does not give one universal price because the total depends on usage across those dimensions. Teams should estimate volume from their own traffic, evaluation frequency, and trace data before setting a rollout budget.
What the announcement does—and does not—claim
The proposal is a measurement and change-management method, backed by Grafana’s Agent Observability capabilities. The cited official material does not report an independent benchmark or a customer-outcome statistic proving that one target, evaluator, or window is optimal. Treat the 95% fulfillment figure as an illustration, validate evaluators against your own conversations, and adjust objectives when the cost of a miss differs between behaviors.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
