You could reduce this to a single point probability. The math is a bit complicated but I think you'd end up with a point probability on the order of 0.001% (~10x lower than the original probability). But if I understand correctly, you aren't actually claiming to have a 0.001% credence.

Yeah, I’m saying the probability is significantly less than 0.02% without saying exactly how much less — that’s much harder to pin down, and there are diminishing returns to exactitude here — so that means it’s a range from 0.00% to <0.02%. Or just <0.02%.

The simplest solution, and the correct/generally recommended solution, seems to be to simply express the probability, unqualified.

Thank you. Karma downvotes have ceased to mean anything to me.

People downvote for no discernible reason, at least not reasons that are obvious to me, nor that they explain. I'm left to surmise what the reasons might be, including (in some cases) possibly disagreement, pique, or spite.

Neutrally informative things get downvoted, factual/straightforward logical corrections get downvoted, respectful expressions of mainstream expert opinion get downvoted — everything, anything. The content is irrelevant and the tone/delivery is irrelevant. So, I've stopped interpreting downvotes as information.

Maybe this is a misapplication of the concept of confidence intervals — math is not my strong suit, nor is forecasting, so let me know — but what I had in mind is that I'm forecasting a 0.00% to 0.02% probability range for AGI by the end of 2034, and that if I were to make 100 predictions of a similar kind, more than 95 of them would have the "correct" probability range (whatever that ends up meaning).

But now that I'm thinking about it more and doing a cursory search, I think with a range of probabilities for a given date (e.g. 0.00% to 0.02% by end of 2034) as opposed to a range of years (e.g. 5 to 20 years) or another definite quantity, the probability itself is supposed to represent all the uncertainty and the confidence interval is redundant.

As you can tell, I'm not a forecaster.

That did come across to me when I watched the interview. For example, in my summary:

Sutskever specifically predicts that another 100x scaling of AI models would make a difference, but would not transform AI capabilities.

He was cagey about his specific ideas on the "something important" that "will continue to be missing". He said his company is working on it, but he can’t disclose details.

I find this secrecy to be a bit lame. I like when companies like DeepMind publish replicable research or, better yet, open source code and datasets. Even if you don’t want to go that far, it’s possible to talk about ideas in general terms without giving away the trade secrets that would make them easy to copy.

Most of the startups that have focused primarily on ambitious fundamental AI research — Vicarious and Numenta are the two examples I’m thinking of — have not ended up successfully productizing any of their research (so far). DeepMind’s done amazing work, but the first AI model they developed with major practical usefulness was AlphaFold, six years after their acquisition by Google and ten years after their founding, and they didn’t release a major product until DeepMind merged with Google Brain in 2023 and worked on Gemini. It’s more likely than not that a research-focused startup like Sutskever’s company, Safe Superintelligence, will not have any lucrative, productizable ideas, at least not for a long time, than they will have ideas so great than even just disclosing their general contours will cause other companies to steal away their competitive advantage. 

My guess is that Safe Superintelligence doesn’t yet have any fantastic ideas that OpenAI, DeepMind, and others don’t also have, and the secrecy just as conveniently covers for that fact as it protects the company’s trade secrets or IP.

I don’t think AGI is five times less likely than I did a week ago, I realized the number I had been translating my qualitative, subjective intuition into was five times too high. I also didn’t change my qualitative, subjective intuition of the probability of a third-party candidate winning a U.S. presidential election. What changed was just the numerical estimate of that probability — from an arbitrarily rounded 0.1% figure to a still quasi-arbitrary but at least somewhat more rigorously derived 0.02%. The two outcomes remain logically disconnected.

I agree that forecasting AGI is an area where any sense of precision is an illusion. The level of irreducible uncertainty is incredibly high. As far as I’m aware, the research literature on forecasting long-term or major developments in technology has found that nobody (not forecasters and not experts in a field) can do it with any accuracy. With something as fundamentally novel as AGI, there is an interesting argument that it’s impossible, in principle, to predict, since the requisite knowledge to predict AGI includes the requisite knowledge to build it, which we don’t have — or at least I don't think we do.

The purpose of putting a number on it is to communicate a subjective and qualitative sense of probability in terms that are clear, that other people can understand. Otherwise, its hard to put things in perspective. You can use terms like extremely unlikely, but what does that mean? Is something that has a 5% chance of happening extremely unlikely? So, rolling a natural 20 is extremely unlikely? (There are guides to determining the meaning of such terms, but they rely on assigning numbers to the terms, so we’re back to square one.)

Something that works just as well is comparing the probability of one outcome to the probability of another outcome. So, just saying that the probability of near-term AGI is less than the probability of Jill Stein winning the next presidential election does the trick. I don’t know why I always think of things involving U.S. presidents, but my point of comparison for the likelihood of widely deployed superintelligence by the end of 2030 was that I thought it was more likely the JFK assassination turned out to be a hoax, and that JFK was still alive.^[1]

I initially resisted putting any definite odds on near-term AGI, but I realized a lack of specificity was hurting my attempts to get my message across.

1. ^{^}

   This approach doesn't work perfectly, either, because what if different people have different opinions/intuitions on the probability of outcomes like Jill Stein winning? But putting low probabilities (well below 1%) into numbers has a counterpart problem in that you don't know if you have the same intuitive understanding as someone else of what a 1 in 1,000 chance, a 1 in 10,000 chance, or a 1 in 100,000 chance means with regard to highly irreducibly uncertain events that are rare (e.g. recent U.S. presidential elections), unprecedented (e.g. AGI), or one-off (e.g. Russia ending the current war against Ukraine), and which can't be statistically or mechanically predicted. 
   
   When NASA models the chance of an asteroid hitting Earth as 1 in 25,000 or the U.S. National Weather Service calculates the annual individual risk of being hit by lightning as 1 in 1.22 million, I trust that has some objective, concrete meaning. If someone subjectively guesses that Jill Stein has a 1 in 25,000 chance of winning in 2028, I don't know if someone with a very similar gut intuition about her odds would also say 1 in 25,000, or if they'd say a number 100x higher or lower.
   
   Possibly forecasters and statisticians have a good intuitive sense of this, but most regular people do not.

Slight update to the odds I’ve been giving to the creation of artificial general intelligence (AGI) before the end of 2032. I’ve been anchoring the numerical odds of this to the odds of a third-party candidate like Jill Stein or Gary Johnson winning a U.S. presidential election. That’s something I think is significantly more probable than AGI by the end of 2032. Previously, I’d been using 0.1% or 1 in 1,000 as the odds for this, but I was aware that these odds were probably rounded.

I took a bit of time to refine this. I found that in 2016, FiveThirtyEight put the odds on Evan McMullin — who was running as an independent, not for a third party, but close enough — becoming president at 1 in 5,000 or 0.02%. Even these odds are quasi-arbitrary, since McMullin only became president in simulations where neither of the two major party candidates won a majority of Electoral College votes. In such scenarios, Nate Silver arbitrarily put the odds at 10% that the House would vote to appoint McMullin as the president. 

So, for now, it is more accurate for me to say: the probability of the creation of AGI before the end of 2032 is significantly less than 1 in 5,000 or 0.02%.

I can also expand the window of time from the end of 2032 to the end of 2034. That’s a small enough expansion it doesn’t affect the probability much. Extending the window to the end of 2034 covers the latest dates that have appeared on Metaculus since the big dip in its timeline that happened in the month following the launch of GPT-4. By the end of 2034, I still put the odds of AGI significantly below 1 in 5,000 or 0.02%.

My confidence interval is over 95%. [Edited Nov. 28, 2025 at 3:06pm Eastern. See comments below.]

I will continue to try to find other events to anchor my probability to. It’s difficult to find good examples. An imperfect point of comparison is an individual’s annual risk of being struck by lightning, which is 1 in 1.22 million. Over 9 years, the risk is in 1 in 135,000. Since the creation of AGI within 9 years seems less likely to me than that I’ll be struck by lightning, I could also say the odds of AGI’s creation within that timeframe is less than 1 in 135,000 or less than 0.0007%.

It seems like once you get significantly below 0.1%, though, it becomes hard to intuitively grasp the probability of events or find good examples to anchor off of. 

The NPR podcast Planet Money just released an episode on GiveWell.

The V-JEPA 2 abstract explains this:

A major challenge for modern AI is to learn to understand the world and learn to act largely by observation. This paper explores a self-supervised approach that combines internet-scale video data with a small amount of interaction data (robot trajectories), to develop models capable of understanding, predicting, and planning in the physical world. We first pre-train an action-free joint-embedding-predictive architecture, V-JEPA 2, on a video and image dataset comprising over 1 million hours of internet video. V-JEPA 2 achieves strong performance on motion understanding (77.3 top-1 accuracy on Something-Something v2) and state-of-the-art performance on human action anticipation (39.7 recall-at-5 on Epic-Kitchens-100) surpassing previous task-specific models. Additionally, after aligning V-JEPA 2 with a large language model, we demonstrate state-of-the-art performance on multiple video question-answering tasks at the 8 billion parameter scale (e.g., 84.0 on PerceptionTest, 76.9 on TempCompass). Finally, we show how self-supervised learning can be applied to robotic planning tasks by post-training a latent action-conditioned world model, V-JEPA 2-AC, using less than 62 hours of unlabeled robot videos from the Droid dataset. We deploy V-JEPA 2-AC zero-shot on Franka arms in two different labs and enable picking and placing of objects using planning with image goals. Notably, this is achieved without collecting any data from the robots in these environments, and without any task-specific training or reward. This work demonstrates how self-supervised learning from web-scale data and a small amount of robot interaction data can yield a world model capable of planning in the physical world.

Again, the caveat here is that this is Meta touting their own results, so I take it with a grain of salt.

I don't think higher scores on the benchmarks mentioned automatically imply progress on the underlying technical challenge. It's more about the underlying technical ideas in V-JEPA 2 — Yann LeCun has explained the rationale for these ideas — and where they could ultimately go given further research.

I'm very skeptical of AI benchmarks in general because I tend to think they have poor construct validity, depending how you interpret them, i.e., insofar as they attempt to measure cognitive abilities or aspects of general intelligence, they mostly don't measure those things successfully. 

The clearest and crudest example to illustrate this point is LLM performance on IQ tests. The naive interpretation is that if an LLM scores above average on an IQ test, i.e., above 100, then it must have the cognitive properties a human does when they score above average on an IQ test, that is, such an LLM must be a general intelligence. But many LLMs, such as GPT-4 and Claude 3 Opus, score well above 100 on IQ tests. Are GPT-4 and Claude 3 Opus therefore AGIs? No, of course not. So, IQ tests don't have construct validity when applied to LLMs if you think IQ tests measure general intelligence for AI systems.

I don't think anybody really believes IQ tests actually prove LLMs are AGIs, which is why it's a useful example. But people often do use benchmarks to compare LLM intelligence to human intelligence based on similar reasoning. I don't think the reasoning is any more valid with those benchmarks than it is for IQ tests.

Benchmarks are useful for measuring certain things; I'm not trying to argue with narrow interpretations. I'm specifically arguing with the use of benchmarks to put general intelligence on a number line, such that a lower score on a benchmark means an AI system is further away from general intelligence and a higher score means it is closer to general intelligence. This isn't valid with IQ tests and it isn't valid with most benchmarks. 

Researchers can validly use benchmarks as a measure of performance, but I want to ward against the overboard interpretation of benchmarks, as if they were scientific tests of cognitive ability or general intelligence — which they aren't.

Just one example of what I mean: if you show AI models an image of a 3D model of an object, such as a folding chair, in a typical pose, they will correctly classify the object 99.6% of the time. You might conclude: these AI models have a good visual understanding of these objects, of what they are, of how they look. But if you just rotate the 3D models into an atypical pose, such as showing the folding chair upside-down, object recognition accuracy drops to 67.1%. The error rate increases by 82x from 0.4% to 32.9%. (Humans perform equally well regardless of whether the pose is typical or atypical — good robustness!) 

Usually, when we measure AI performance on some dataset or some set of tasks, we don't do this kind of perturbation to test robustness. And this is just one way you can call the construct validity of benchmarks into question. (If benchmarks are being construed more broadly than their creators probably intend, in most cases.)

Economic performance is a more robust test of AI capabilities than almost anything else. However, it's also a harsh and unforgiving test, which doesn't allow us to measure early progress.

Possibly something like V-JEPA 2, but in that case I'm just going off of Meta touting its own results, and I would want to hear opinions from independent experts.