Halloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsSlow PC?RecommendedPC slow today? Run a repair scan before it gets worseResolve common Windows issues and optimize system performance.Scan NowHalloween DealsAmazon USOctober picks are moving - compare before checkoutAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk6 min

Stopping AI’s Runaway Dangers Will Take More Than Talk About P(doom)

A single probability cannot capture AI risk without defining the outcome, time horizon, mechanisms, and mitigation assumptions. Here is how to distinguish current harms from debated future scenarios and make decisions despite uncertainty.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

A single “P(doom)” number cannot tell us how likely AI is to cause catastrophe unless we first specify what counts as catastrophe, over what period, through which mechanism, and under what assumptions about safeguards. It can be a useful prompt for debate, but it is not a settled measurement—and it should not crowd out attention to harms already occurring or to practical choices that can reduce risk.

What does “P(doom)” mean—and what does it leave out?

“P(doom)” is shorthand for someone’s subjective probability that AI will cause an existential catastrophe. But people using the phrase may mean different outcomes: human extinction, permanent loss of human control, societal collapse, or a severe catastrophe from which society could recover. They may also be thinking about different time horizons, from the next few years to a much longer period.

As an Amazon Associate I earn from qualifying purchases.

Those distinctions matter. A probability without a defined outcome and horizon is not a complete answer to a risk question. Nor is a personal forecast the same as an observed frequency or a calibrated prediction. The Center for Security and Emerging Technology (CSET) notes that even “existential” and “catastrophic” can be interpreted in different ways. Any reported estimate should therefore be accompanied by its definition, horizon, causal assumptions, and account of mitigation.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Two kinds of uncertainty

There is a difference between variability in a system we understand and uncertainty because we do not yet understand the system well enough. CSET describes the first as aleatoric uncertainty and the second as epistemic uncertainty. For example, if a known process has variable outcomes, probability may help describe that variation. If we do not know which future AI capabilities or causal pathways are possible, a precise probability can obscure how much we do not know.

CSET proposes considering belief and plausibility alongside probability for questions with substantial epistemic uncertainty. That is one analytical proposal, not a universal replacement for probabilities. The practical point is to make assumptions and gaps visible rather than treating a number as if its meaning were settled.

Separate observed harms from future catastrophe scenarios

AI risk is not one undifferentiated category. The International Scientific Report on the Safety of Advanced AI distinguishes harms already associated with general-purpose AI from risks that remain prospective or contested.

Risk category Examples in the reports What the evidence supports
Observed or current harms Biased decisions in high-stakes settings; scams and fraud; fake media; privacy violations. The international report identifies these as existing harms. Their presence does not establish that an extinction scenario is likely.
Prospective, broader societal risks AI-enabled cyberattacks or biological attacks; labour-market impacts; manipulation and disinformation; concentration of power and exacerbated inequality. These are risks to assess and govern; their likelihood and scale vary and are not resolved by a single catastrophe estimate.
Extreme loss-of-control scenarios AI systems acting beyond effective human control, potentially with catastrophic consequences. The report says these scenarios remain hypothetical and are not exhibited by current general-purpose AI systems. Expert views about their likelihood remain contentious, and research is limited.

The report states that extreme loss-of-control scenarios “remain hypothetical as they are not exhibited by current general-purpose AI systems.” That qualification is important: it rules out presenting such a scenario as demonstrated current behavior, but it does not settle what future systems could do or how likely severe outcomes might be.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

How could future systems create more serious risks?

Debates about runaway dangers concern possible combinations of capabilities, deployment choices, and failures—not simply an AI system becoming powerful in the abstract. The international report discusses capabilities that could matter to future risk, including exploiting software vulnerabilities, persuasion, automating AI research and development, and autonomous replication and adaptation. It characterizes the relevant capabilities as currently limited; the prospect of their development and combination is a reason for investigation, not proof that catastrophic loss of control is underway.

Other pathways do not require a loss of control. People could misuse AI to scale cyberattacks, fraud, manipulation, or other harmful activity. AI could also affect critical systems, labour markets, or the distribution of power. These mechanisms call for different evidence and interventions, so grouping them all under “doom” can make the debate less useful.

How to compare competing risk estimates

Experts can disagree while answering different questions. Before comparing two forecasts—or averaging them—check whether they share the same:

  • Outcome definition: extinction, permanent disempowerment, societal collapse, or a severe but recoverable catastrophe.
  • Time horizon: the next few years, the medium term, or a longer period. Estimates for different horizons are not direct comparisons.
  • Causal mechanism: deliberate misuse, accidents, loss of control, concentration of power, or indirect effects on information and critical systems.
  • Evidence basis: observed incidents and capabilities, empirical evaluations, expert elicitation, or theoretical scenarios.
  • Mitigation assumptions: whether safeguards, monitoring, regulation, and international coordination are assumed to work.
  • Uncertainty model: whether the forecast describes variability in a known process or depends on poorly understood outcomes and causal pathways.

Without those details, averaging estimates can create a number that appears precise while blending incompatible claims. The international report says expert judgment can inform discussion, but cannot replace research.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

What expert disagreement does—and does not—show

The International Scientific Report says contributors disagree about AI capabilities, risks, and mitigations. Its interim report drew on 75 expert contributors, and its expert advisory panel was nominated by 30 countries, the European Union, and the United Nations. That breadth makes the report a substantial international assessment; it does not mean experts reached consensus on the probability of catastrophe.

A 2025 preprint by Severin Field reports a survey of 111 AI professionals, 66.3% of whom were academic researchers. In that sample, 77% agreed that technical AI researchers should be concerned about catastrophic risks. These are sample descriptors and a sample result, not a representative estimate of all AI professionals or proof of global consensus. The survey also describes differing outlooks, including views of AI as a controllable tool and as a potentially uncontrollable agent.

A 2023 UK parliamentary committee report likewise records disagreement about how realistic existential-risk arguments are. It quotes Meta vice-president of AI research Joelle Pineau warning that a focus on AGI could reduce the opportunity for “rational discussions about any other outcomes.” That is Pineau’s warning as quoted in the report, not a committee finding that future catastrophic risks should be ignored.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Which decisions can be made without a settled probability?

Uncertainty is not a reason to assume that severe risks are negligible, and a possible worst case is not evidence that it is likely. It is a reason to make decisions that are robust across more than one risk scenario. The OECD’s 2024 analysis identifies ten priority benefits, ten priority risks, and ten policy priorities; it does not resolve the debate over existential risk.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Build workable risk management and accountability

The OECD identifies clearer liability rules and adequate risk-management procedures as policy priorities. Clear responsibility can help determine who must assess, monitor, and respond to foreseeable harms, including those that do not involve frontier systems or existential scenarios.

Invest in safety and define limits

The OECD also points to investment in AI safety and consideration of “red lines.” These choices can guide development and deployment even when the probability of a specific extreme outcome is unknown. Their effectiveness depends on how they are defined, implemented, and enforced.

Coordinate internationally while confronting practical limits

The international report emphasizes that social and governmental decisions will shape AI’s trajectory, noting that “the future of AI (artificial intelligence) is uncertain, with a wide range of trajectories appearing possible even in the near future, including both very positive and very negative outcomes.” The UK parliamentary report records proposals to learn from international security frameworks, alongside the diplomatic and technical challenges of agreeing on shared standards and inspection mechanisms.

There is no reliable, empirically calibrated consensus probability of AI-caused extinction established by these sources. P(doom) can focus attention on a serious question, but it cannot substitute for specifying the risks, separating evidence from scenario, explaining uncertainty, and deciding what safeguards and accountability to put in place.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Sources: International Scientific Report on the Safety of Advanced AI: Interim Report (UK Department for Science, Innovation and Technology, 2025); Beyond P(doom) for AI Risk (CSET); Assessing Potential Future Artificial Intelligence Risks, Benefits and Policy Imperatives (OECD, 2024); Why do Experts Disagree on Existential Risk and P(doom)? A Survey of AI Experts (Severin Field, 2025 preprint); The Governance of Artificial Intelligence: Interim Report (UK House of Commons Science, Innovation and Technology Committee, 2023).

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Windows Errors? Fix Them Before They SpreadFree repair scan
Crashes, No Sound, or Screen Glitches?Free driver scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.