Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →METR’s time horizon is a calibrated task-difficulty measure, not a stopwatch for autonomous operation. It estimates the human-expert time needed for a task and then identifies the duration at which an AI agent is predicted to succeed with a stated reliability. A model with a two-hour T50 reaches a fitted 50% success probability on tasks whose human-estimated completion time is about two hours; it is not expected to work independently for two hours or finish every two-hour task.
To interpret a METR result responsibly, read the version, reliability level, task domains, human baseline, success rule, repeated-run protocol and uncertainty together. The public Time Horizon 1.1 (TH1.1) measurements also carry a specific warning: values above 16 hours are unreliable with the current task suite.
As an Amazon Associate I earn from qualifying purchases.
What METR’s time horizon actually measures
METR defines the task-completion time horizon as “the task duration (measured by human expert completion time) at which an AI agent is predicted to succeed with a given level of reliability.” The duration is therefore a human-calibrated difficulty scale. METR is asking, in effect: At what estimated human task length does this agent’s probability of meeting the benchmark’s success rule fall to a chosen level?
The two labels most readers see are:
- T50: the human-estimated task duration where the fitted agent success probability is 50%.
- T80: the duration where the fitted curve reaches 80% success.
A T50 is easier to estimate than a high-reliability horizon because the data usually contain more observations near the middle of the success curve. A T80 requires the agent to succeed more consistently, so it can be more sensitive to sample size, broken tasks and other methodological choices.
What a two-hour result means
Suppose a result says an agent has a two-hour T50 on a specified METR suite. The defensible reading is: on those specified tasks, using that version, scaffold and scoring procedure, the fitted success curve reaches 50% at a human-estimated duration of approximately two hours.
It does not mean the agent can:
- operate continuously for two wall-clock hours;
- complete every task that takes a person two hours;
- perform any kind of intellectual or physical work that a person could do in two hours;
- replace a worker whose job contains two-hour activities; or
- solve half of every task in a two-hour bucket.
Agent runtime can be shorter or longer than the human estimate. The metric is about the difficulty of the task, not the time the AI spends operating.
How METR produces the number
1. METR defines a checkable task
METR favors coherent, self-contained tasks that cannot be trivially split into independent subtasks. The main domains are software engineering, machine learning and cybersecurity, with some general-reasoning tasks. Each task has a success rule: usually a binary outcome or a threshold applied to a continuous score.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Scoring is designed to be checkable and difficult to game. That makes results more reproducible than an open-ended judgment about whether an agent’s work “looks good,” but it also means the benchmark may be cleaner and narrower than workplace work.
2. METR estimates human completion time
Skilled professionals receive the same task instructions and affordances as the agents. For most tasks, METR uses the geometric mean of successful human completion times. The geometric mean is appropriate for durations that can vary multiplicatively: one unusually slow or fast completion does not dominate the estimate as strongly as it would under a simple arithmetic mean.
When successful human baselines are unavailable, METR uses expert forecasts or estimates derived from quality-assurance data. The HCAST methodology illustrates why the source matters: 140 individuals attempted most of 189 tasks; 110 tasks had at least one successful human baseline, while 79 task times were forecasted.
These are generally low-context estimates. A participant receives limited prior knowledge of the organization, codebase or institutional history. A task estimated at two hours may therefore resemble something a capable contractor or new hire could complete in two hours, not something an experienced insider could do after years of accumulated context.
3. METR runs the agent repeatedly
METR combines a model with an agent scaffold, such as Modular or Triframe, and provides tools and a task environment. The current FAQ describes six independent runs per task on the test set. After runs finish, METR performs resource checks and reviews for reward hacking or other invalid behavior.
Repeated attempts matter. A single successful demonstration can show that an agent is capable of one solution path; it does not establish the reliability needed for a time horizon. The horizon comes from the pattern of successes and failures across tasks and runs.
4. METR fits a success curve
METR fits a logistic curve linking estimated human task duration to agent success probability. The reported horizon is the duration at which that fitted curve crosses 50%, 80% or another chosen reliability level.
The curve is a statistical summary of the benchmark data, not a hard boundary in the agent’s abilities. Individual tasks can sit well above or below it. A task longer than a model’s T50 can still be solved, and a shorter task can still fail.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →How to read a METR chart
Read the version before comparing numbers
The public chart distinguishes Time Horizon 1.1 from the original Time Horizon 1.0 calculations. METR’s research index dates the TH1.1 release to January 29, 2026. Changes in task composition, scoring or other methodology can move a model’s reported value even when the underlying model has not changed.
Compare values within the same version and protocol unless the chart clearly documents a cross-version adjustment. Never present a TH1.0 value and a TH1.1 value as a clean trend line without labeling the change.
Identify the reliability level
Check whether the point is T50, T80 or another threshold. A T50 answers “where does the fitted curve reach half-success?” A higher-reliability horizon answers a stricter question and will normally be shorter. Calling a T50 an “80%-reliable capability” reverses the meaning of the metric.
METR notes that estimating a 99% horizon would require many more short tasks and would be especially sensitive to broken tasks and methodological decisions. A chart point at a high reliability level should therefore be read with its uncertainty, not as a guaranteed operating limit.
Inspect uncertainty and overlap
Point estimates are fitted summaries. Use confidence intervals or bootstrap displays when the chart supplies them. If two nearby models have overlapping intervals, their rank may not be stable; a small visual separation is not proof of a meaningful capability difference.
Uncertainty can arise from the number of tasks, the number of independent runs, the spread of human times, the fitted-curve specification and exclusions for invalid or reward-hacked runs. A precise-looking decimal does not remove those sources of variation.
Look at the task distribution
The suite is concentrated in software engineering, machine learning and cybersecurity. A result should not be generalized automatically to writing, management, sales, legal work, research that depends on institutional context, physical tasks or socially interactive jobs.
Within HCAST, agent performance declines as estimated human duration rises, with a marked drop between the 15-minute-to-1-hour bucket and the 1-to-4-hour bucket. That gradient supports human-calibrated duration as an interpretable capability axis, but it remains a result about the tested task families.
Check the human baseline and context
Human time is not a universal unit of work. It depends on expertise, tools, prior context and the definition of “done.” METR’s low-context setup can overstate the time a familiar professional would need in a real organization. Conversely, a real task may involve meetings, approvals, access requests or tacit knowledge that the benchmark omits.
When quoting a value, state that it uses METR’s human-estimated duration under the relevant protocol, rather than calling it “hours of work the AI can do.”
Read the success rule
Automatically scored, well-specified tasks generally give cleaner signals. Holistic scoring and messier tasks can produce lower performance because success involves judgment, completeness or practical usability. Benchmark success should not be translated directly into job automation without examining what the score required.
Apply the long-horizon warning
METR currently states: “Measurements above 16 hrs are unreliable with our current task suite.” Treat any value above that point as a warning about measurement quality, not as a dependable claim about a full workday of autonomy.
What the results show—and what they do not
What they show
- Task difficulty can be placed on a common, human-time-calibrated axis.
- Across the HCAST tasks, success generally falls as estimated human duration rises.
- Repeated agent attempts and a fitted curve provide a more informative reliability estimate than a single demo.
- Versioned, task-specific measurements can support comparisons between models tested under the same conditions.
What they do not show
- They do not measure continuous autonomous operating time.
- They do not estimate the share of every job that an AI can automate.
- They do not prove that all tasks of a given duration have the same difficulty.
- They do not transfer automatically from software, ML or cybersecurity to every occupation.
- They do not remove the need for human oversight, context, coordination or verification.
Most jobs combine many task types and include tacit context, interpersonal coordination and success criteria that are difficult to score algorithmically. A benchmark horizon is consequently one input into an automation assessment, not a job-replacement forecast.
How to compare two METR results correctly
Use the following checklist before describing one result as higher, lower or evidence of progress:
| Comparison axis | What to verify | Why it matters |
|---|---|---|
| Version | TH1.0 or TH1.1, and release date | Task composition and scoring may have changed. |
| Model and scaffold | The model, agent harness and tool setup | The scaffold can materially affect success. |
| Task mix | Domains, task families and duration distribution | A different suite can change the apparent horizon. |
| Reliability threshold | T50, T80 or another level | Different thresholds answer different questions. |
| Human baseline | Successful times, forecasts or QA-derived estimates; context provided | The time axis is only as meaningful as its calibration. |
| Sample and repeats | Number of tasks and independent runs per task | Small samples make the fitted curve less stable. |
| Uncertainty | Confidence intervals or bootstrap ranges | Overlapping intervals can make rankings uncertain. |
| Validity checks | Resource limits, reward-hack review and invalid-run exclusions | Unfiltered or exploited runs can inflate apparent performance. |
Only after these axes match should a difference be described as a model improvement rather than a protocol or measurement change.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.A worked interpretation without overclaiming
Imagine a TH1.1 chart showing an agent at a T50 of roughly two hours on the published software-and-ML task mix. A careful report would say:
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minute“On METR’s TH1.1 tasks and scaffold, the fitted success curve reaches 50% at a human-estimated duration of about two hours. The result concerns the benchmark’s specified, low-context tasks; it is not a prediction that the agent can operate autonomously for two hours or automate two-hour workplace activities.”
Best Value
The report should add the chart’s uncertainty, the date or release, and the exact reliability level. If the value is above 16 hours, it should also carry METR’s warning that measurements beyond that point are unreliable with the current suite.
Common misreadings and their fixes
“The AI can do two hours of work.”
Fix: Say that the fitted 50% success point corresponds to a human-estimated task duration of about two hours under the stated benchmark conditions.
“An eight-hour horizon means job replacement.”
Fix: Examine the job’s task mix, context, coordination, error costs and oversight requirements. A benchmark duration is not a labor-market forecast.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errors“The model solved a task, so its horizon is at least that long.”
Fix: Look at repeated independent runs across the suite. The metric is a reliability estimate, not the longest successful demo.
“A higher plotted point always means a better model.”
Fix: Confirm matching version, scaffold, task mix, threshold and uncertainty. Overlapping intervals or a protocol change may explain the visual difference.
“Human time is objective ground truth.”
Fix: Identify whether the value came from successful human completions, expert forecasts or QA-derived estimates, and ask how much context participants had.
Using METR results in an evaluation or deployment decision
METR is most useful when treated as one layer in a broader evaluation. A practical sequence is:
Free tools Windows power users keep installed
One-click scans. No signup required.
- Define the real work unit. Break the proposed use case into tasks with explicit deliverables, dependencies and acceptable error rates.
- Match the task family. Determine whether the benchmark resembles the domain, tools, data access and context of the real work.
- Choose the required reliability. A 50% success rate may be unsuitable for production actions that are costly or unsafe to redo.
- Measure end-to-end performance. Include setup, context retrieval, review, correction, handoffs and approval time, not just benchmark execution.
- Set a verification policy. Decide which outputs require automated tests, human review, sandboxing or dual approval.
- Track failures by type. Separate misunderstanding, tool misuse, missing context, scoring ambiguity and reward-hacking behavior.
- Re-evaluate after protocol changes. A new scaffold, tool permission, model version or task distribution can change the result.
This approach preserves what the horizon measures—a useful relationship between task difficulty and success—without turning it into a claim the benchmark was not designed to make.
Reproducibility and source handling
METR provides analysis code and raw data from its time-horizon materials, and the public GitHub repository describes extracting a threshold-crossing time horizon from the fitted success relationship. The HCAST paper documents task-family construction, human-baseline collection, contamination controls and the Modular and Triframe scaffolds.
When reproducing a chart, use the versioned source and raw data that belong together. Label TH1.0 and TH1.1 separately, preserve the task and reliability definitions, and record any exclusions for invalid or reward-hacked runs. Mixing values from different versions can create an apparent trend that is actually a measurement change.
Quick Recap
Key takeaways
- A METR T50 is a human-time-calibrated difficulty threshold at 50% predicted success; T80 uses the same idea at 80%.
- The number is not a wall-clock autonomy duration.
- Results are driven by task selection, human baselines, scaffolds, repeated runs, scoring and logistic-curve fitting.
- The public suite is mainly software engineering, machine learning and cybersecurity, so transfer to other work is limited.
- Low-context human estimates may differ substantially from the speed of an experienced workplace insider.
- Compare only like with like, and inspect uncertainty before ranking models.
- TH1.1 measurements above 16 hours are flagged by METR as unreliable with the current task suite.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Recommended Free Tools
