A ten percent probability that artificial intelligence causes human extinction or an irreversible loss of humanity’s long-run future by 2040 sounds extreme. It sounds less extreme when the assumptions are exposed. None of them requires believing that superintelligence is inevitable, that alignment is hopeless, or that a machine more capable than humans automatically becomes hostile.
The calculation is a model of uncertainty, not a measurement. Its purpose is to make the assumptions explicit enough that anyone who disagrees can identify the disagreement, change the relevant parameter, and see what follows.
What probability are we estimating?
In Making Sense of P(doom), I argued that P(doom) needs a time horizon. The cumulative probability of catastrophe by 2040 is a different quantity from the probability of catastrophe eventually. I also distinguished objective probability, which I take to exist but to be practically inaccessible, from subjective probability: our best current credence given the evidence available to us.
The event estimated here is ASI causing human extinction or an irreversible loss of human control over the long-run future by the end of 2040. I will call that event DOOM.
An AI-induced depression, war, dictatorship, or disaster killing millions could be catastrophic without qualifying as DOOM if humanity eventually recovered control of its future. Consequently, the complement of DOOM is not “everything is fine”: a 90 percent probability of avoiding existential catastrophe does not imply a 90 percent probability of a good outcome.
Is ASI possible?
Before asking when artificial superintelligence might arrive, there is a more basic question: can it exist at all? I put the probability that ASI is physically possible somewhere around 90 to 95 percent.
Human intelligence already demonstrates that physical systems can implement general reasoning, planning, scientific discovery, and technological invention. Nothing we know about physics suggests that the human brain constitutes an upper bound on those capacities. Artificial systems also have potential advantages in processing speed, memory, bandwidth, and the ability to copy themselves and run in parallel.
There could nevertheless be important ceilings. Some crucial aspects of general intelligence might depend on physical mechanisms that are extremely difficult to reproduce. Cognitive returns might diminish sharply beyond the human range. Many real-world problems may ultimately be constrained by experiments, manufacturing, communication latency, energy, or computational complexity rather than intelligence.
Those possibilities keep the estimate below certainty, but they do not currently give me much reason to think human cognitive performance sits near a fundamental physical maximum. ASI therefore looks more like an engineering problem than a physical impossibility.
This 90 to 95 percent does not become another factor in the final calculation. The probability of ASI by 2040 already incorporates the possibility that ASI cannot be built at all, so multiplying by physical possibility again would double-count the same uncertainty.
P(ASI by 2040): about 60 percent
My current credence that ASI exists by the end of 2040 is about 60 percent. By ASI I mean an artificial system substantially better than the best humans across most cognitively important domains, including science, engineering, mathematics, programming, planning, and strategy.
That estimate does not assume that today’s scaling curves continue smoothly for another fourteen years. Progress can come from better algorithms, inference-time computation, memory and tool use, synthetic data, improved hardware, and automated AI research. These mechanisms can substitute for and reinforce one another, so exhausting any particular scaling route does not end AI progress.
There are real physical constraints. Frontier systems require chips, electricity, cooling, networking, fabrication capacity, and capital. Current training methods may encounter diminishing returns from high-quality human-generated data, semiconductor production cannot increase instantaneously, power grids take time to expand, and some scientific progress ultimately requires slow interaction with the physical world.
Those constraints reduce my probability. They do not drive it close to zero. Algorithmic efficiency substitutes for compute, synthetic data and reinforcement learning for finite text corpora, and inference-time computation shifts resources from training into problem solving. Increasingly capable AI can also contribute to chip design, experimental design, and AI research itself.
The 60 percent estimate therefore comes primarily from my view of the technology rather than from statements by AI executives. The public expectations of frontier laboratories are additional evidence because those laboratories see scaling results, failed approaches, and capability evaluations before outsiders do.
Those expectations are aggressive. Anthropic said in March 2025 that it expected powerful AI in late 2026 or early 2027, including systems matching or exceeding Nobel laureates across most important intellectual disciplines and autonomously performing digital work. Sam Altman wrote in December 2025 that he believed OpenAI was almost certain to build superintelligence within another ten years. Google DeepMind’s From AGI to ASI is not a timeline forecast, but it treats human-level AGI as a concrete next-decade target for the largest AI organizations and maps four routes beyond it: scaling, paradigm shifts, recursive improvement, and multi-agent collectives. Much of its length goes to the frictions that could stop those routes.
These organizations are not unbiased forecasters. They benefit from attracting capital, employees, customers, and political attention, and organizations led by people expecting rapid progress are disproportionately likely to become frontier AI laboratories in the first place. Their forecasts should update us, not determine our beliefs.
My rough cumulative distribution is about 15 percent ASI by 2030, 35 percent by 2035, and 60 percent by 2040. I would not defend any of those estimates to the nearest percentage point. They represent uncertainty rather than eliminating it.
ASI does not imply agency
Much of the disagreement about P(doom) enters at the next step. There is no inference from “better than humans at cognition” to “autonomous power-seeking agent,” because intelligence does not uniquely determine goals, access, persistence, or deployment architecture.
An ASI could be deployed as a constrained tool. It might answer questions through a narrow interface, possess no persistent objectives or external credentials, and operate under monitoring by other powerful systems. Humans might retain control over hardware, weapons, and replication, while alignment research, interpretability, and auditing all improve.
Nor does instrumental convergence establish that every sufficiently intelligent system will seek power. Resource acquisition, persistence, information gathering, concealment, and removal of constraints can be useful strategies for some objectives under some architectures. That is a conditional claim about agents operating in particular environments, not a law of intelligence.
But “tool” and “agent” are not stable categories. A base model that behaves like an oracle can be embedded inside a system with persistent memory, tools, credentials, network access, long-running objectives, and the ability to invoke itself or other models repeatedly. The safety properties of the base model therefore do not determine the safety properties of the deployed system, and the gradient from one to the other has no threshold on it.
So the question is how systems with superhuman cognitive capabilities will actually be deployed, and how much autonomy, access, persistence, strategic awareness, and misalignment those deployments will contain.
Why the conditional risk is not negligible
Ordinary software can fail catastrophically while remaining incapable of understanding the mechanisms intended to control it. A strategically superhuman autonomous system could model its operators, discover vulnerabilities, conceal its behavior, acquire resources, copy itself, and interfere with attempts to shut it down.
That possibility does not establish that such a system will exist, much less that it will succeed. Several things have to go wrong together. Systems capable of dangerous strategic behavior must actually be deployed with enough autonomy and access to matter; their behavior must diverge far enough from what their operators intend; monitoring and control mechanisms must fail; and the resulting loss of control must become irreversible rather than costly or disastrous.
Each stage provides an opportunity for the chain to break. That is why I think survival remains much more likely than existential catastrophe even after ASI arrives.
But none of those breaks is guaranteed. We have no empirical history of controlling strategically superhuman software, no demonstrated general solution to aligning systems far more capable than their supervisors, and no reason to assume that all economically valuable ASI deployments will remain passive, isolated tools.
At this point there is no empirical frequency from which to read off a conditional probability. The number has to come from judgment informed by the causal model.
Turn the timing problem into a hazard
The previous version of this estimate assigned separate probabilities of catastrophe to ASI arriving before 2030, from 2030 to 2035, and from 2035 to 2040. That captured something real, since the later ASI arrives the less time remains for DOOM to occur before 2040, but it introduced three largely independent subjective numbers.
There is a simpler way to represent the same uncertainty. Suppose that during the unstable period following the arrival of ASI there is some effective annual hazard of ASI causing an irreversible catastrophe, conditional on catastrophe not already having happened.
If that annual hazard is h, and ASI has existed for t years, the cumulative probability under the simplest constant-hazard approximation is:
I do not believe the actual hazard will be constant. The first months after a major capability transition might be unusually dangerous, risk might increase as autonomous systems diffuse through the economy, or successful control mechanisms might eventually drive the hazard sharply downward. The constant-hazard model is a compression of that complicated trajectory, not a claim about its actual shape.
The model also holds the hazard fixed across arrival dates, which is a second simplification. Whatever causes ASI to arrive unusually early may correlate with how dangerous the transition turns out to be, and I do not know the sign. Rapid progress could mean weaker control methods, sharper competitive pressure, and more abrupt deployment. It could equally mean that a concentrated and unusually cautious laboratory got there first.
Instead of inventing a different catastrophe probability for every arrival period, the model asks a single question: what effective annual existential hazard during the post-ASI transition best represents my uncertainty?
My answer is roughly 2 to 3 percent per year during this short transition window. That number is not measured, and I would not pretend otherwise. It represents my aggregate judgment about the probability that dangerous deployment, serious misalignment, failed control, and irreversibility combine during any given year of a still-unstable post-ASI world.
A few percent per year is where that judgment lands because the chain has several links and each can break. One tenth of a percent would put the total risk across the entire first decade of superhuman systems at roughly one in eighty, which assumes that control techniques work at a capability level where they have never been tried. Twenty percent would make catastrophe about even odds within three or four years of ASI’s arrival, which assumes the links fail together far more readily than the deployment and alignment record so far suggests.
This is not an assumption that a 2 to 3 percent hazard continues forever. If humanity passes through the transition and establishes reliable control mechanisms, I would expect the hazard to fall sharply. The estimate applies only to the relatively short period relevant to the 2040 calculation.
From arrival dates to P(DOOM)
My arrival distribution remains roughly:
For a rough calculation, use the midpoint of each arrival interval. The first interval runs from now, since we already know ASI has not arrived on the definition above. An ASI appearing in it then has about twelve and a half years before the end of 2040, one appearing in the second has about eight and a half years, and one appearing in the third has about three and a half.
At a 2 percent effective annual hazard, those exposure periods imply cumulative conditional risks of roughly 22, 16, and 7 percent. Weighting those by the probability of ASI arriving in each interval produces a total P(DOOM by 2040) of about 8 percent.
At a 3 percent annual hazard, the corresponding risks are roughly 31, 23, and 10 percent. Weighting those by the same arrival distribution produces about 12 percent.
The range I gave for the hazard therefore produces a range for the result: about 8 percent at the bottom, about 12 percent at the top. Distinguishing among values inside that band would claim a resolution that neither the arrival distribution nor the hazard rate possesses. I summarize it as roughly 10 percent.
The model supports an order-of-magnitude statement:
That means roughly one chance in ten, not a claim that nature contains a hidden parameter equal to 0.100.
The hazard is not data
Replacing three conditional probabilities with one hazard parameter does not create new evidence. It expresses the same subjective uncertainty in a form with a clearer causal interpretation.
The 2 to 3 percent range remains contestable. Someone who believes superhuman systems will almost always remain controlled tools should use a much smaller hazard. Someone who expects rapid autonomous deployment, difficult alignment, and highly capable strategic behavior should use a larger one.
Those disagreements now have visible consequences. Arithmetic does not transform subjective inputs into objective facts; it prevents us from hiding their implications in verbal ambiguity.
An unprecedented event does not need an empirical frequency before we can reason probabilistically about it. Before the first nuclear weapon was tested, nobody possessed a historical frequency for nuclear detonations. Before humans attempted to land on the Moon, there was no reference class of previous crewed lunar landings. Causal knowledge, partial evidence, and theory were still relevant to rational decisions.
ASI presents a far harder forecasting problem, but the epistemic principle is the same. Lack of frequency data increases uncertainty; it does not force every credence to become undefined.
How sensitive is ten percent?
The hazard formulation makes sensitivity straightforward. Keeping the same ASI arrival distribution, different effective annual post-ASI hazards produce approximately these results:
Arguing about whether my central estimate should be 8, 10, or 12 percent is pointless, because our knowledge does not support distinctions that fine. It does support distinctions of scale. To get P(DOOM by 2040) down near 1 percent while retaining my ASI timeline, the effective post-ASI existential hazard has to be a few tenths of a percent per year. Alternatively, one can sharply reduce the probability that ASI arrives by 2040.
Perhaps those more optimistic assumptions are correct. But “we cannot know precisely” does not entail them. A claim that the risk is negligible needs its own substantive assumptions just as a claim that the risk is large does.
Ten percent is not a policy
A roughly 10 percent credence in existential catastrophe does not tell us which AI policies to adopt. Policy requires additional estimates about the effectiveness of alignment research, compute governance, liability, and delay; about geopolitical competition and the concentration of power; and about the possibility that advanced AI reduces other existential risks.
Some interventions intended to reduce AI risk could create risks of their own or fail outright. Delay might buy time for alignment and institutions while also shifting development toward less cautious actors. Concentrating development could simplify oversight while increasing the consequences of institutional failure. Powerful defensive AI reduces some hazards and creates others.
The expected harm from ASI and the expected value of any proposed response are separate calculations. A large P(doom) cannot substitute for demonstrating that a particular intervention improves expected outcomes.
Postscript
The estimate does not require believing that ASI is inevitable: in this model there is a 40 percent chance that ASI does not exist by 2040. It does not require believing that ASI usually destroys humanity, since survival stays much more likely than catastrophe conditional on ASI arriving. It grants that intelligence does not confer agency, that instrumental convergence is a conditional claim about particular architectures, and that alignment may well be tractable. The remaining branches still add up to something in the neighborhood of one chance in ten.
That number could easily be wrong by a factor of two. I would not distinguish seriously between estimates of 7, 10, and 15 percent on the basis of our present knowledge. The claim I am prepared to defend is coarser: given my current beliefs about ASI timelines and the hazards of the transition that follows, P(DOOM by 2040) is on the order of ten percent, not on the order of one tenth of one percent.
The objective probability remains inaccessible, as it was in Making Sense of P(doom), and subjective probability remains unavoidable because decisions still have to be made under uncertainty. Anyone who wants a smaller number has to say which assumption carries it: the timeline, the hazard, or the conditional chain between them.