No-one knows when AI will begin having transformative impacts upon the world. People aren’t sure and shouldn’t be sure: there just isn’t enough evidence to pin it down.
But we don’t need to wait for certainty. I want to explore what happens if we take our uncertainty seriously — if we act with epistemic humility. What does wise planning look like in a world of deeply uncertain AI timelines?
I’ll conclude that taking the uncertainty seriously has real implications for how one can contribute to making this AI transition go well. And it has even more implications for how we act together — for our portfolio of work aimed towards this end.
By AI timelines, I refer to how long it will be before AI has truly transformative effects on the world. People often think about this using terms such as artificial general intelligence (AGI), human level AI, transformative AI, or superintelligence. Each term is used differently by different people, making it challenging to compare their stated timelines. Indeed even an individual’s own definition of their favoured term will be somewhat vague, such that even after their threshold has been crossed, they might have trouble specifying in which year it happened.
Many commentators have suggested this makes terms such as AGI useless, but I don’t think that is right.
I like to think of it in terms of a group of hikers seeing a mountain in the distance, towering up into the clouds and beyond, with its snowy peak catching the sun’s light. They talk animatedly about how amazing it would be to climb so high that they are inside a cloud. Or imagine being above the clouds, looking over them like an angel. After many hours of climbing, they notice there is a faint haze. Are they inside the cloud now? The mist gradually gets thicker until they can only see 10 metres ahead. Are they inside it now? Then it drops to 9 metres. Then 8. Then visibility starts to increase again. After an hour there is only the slightest haze. Are they above the clouds now? Another 30 minutes and there is no haze, and they can all agree they are above the clouds.
It is clear that at some point they were inside the cloud and sometime later were above it. And it is clear that these were sensible and useful concepts. For example, they took precautions like roping themselves together for the journey through the cloud due to the low visibility and took cameras with them because they knew they could take beautiful photos above the clouds. A lack of sharp boundaries doesn’t make these concepts useless. But they were admittedly a lot more useful when the hikers were on the ground, planning their route, and a lot less useful in the debatable boundary zones.
I think of AGI (and human-level intelligence) as the cloud, and superintelligence as being above the cloud. They are useful concepts, despite their vagueness. But they’re markedly less useful when you get close to them.
So I think that forecasting when we’ll reach some threshold for advanced, game-changing AI makes sense. Albeit there is some inherent uncertainty due to the vagueness of the ideas, and we have to be careful when comparing our estimates to make sure we’re talking about the same version of these concepts.
Regarding AGI, it’s already getting a bit misty. In February there was a piece in Nature arguing that the current level of frontier AI should count as AGI. I’d set the bar a bit higher than that, but I agree it is already debatable whether we’re in the cloud.
For my purposes, I think the key threshold is when the system is capable enough that there are dramatic changes to the world — civilisational changes. For example, the point where AI could take over from humanity were it misaligned, or it has made 50% of people permanently unemployable, or has doubled the global rate of technological progress. Something like that. The reason I pick this point is that I think it is the one that matters most for decision-relevant planning of our strategies and careers. For many purposes we’d want our plans to pay off before we reach that point, and plans that reach fruition afterwards are likely to be significantly disrupted. I’ll refer to this as transformative AI and will make sure to show what rubric other people are using when they give their own timeline numbers.
Discussions about timelines are usually framed as a debate between short timelines vs long timelines.
One of the most prominent supporters of very short timelines is Dario Amodei, CEO of Anthropic. In January 2025 he said:
Making AI that is smarter than almost all humans at almost all things will require millions of chips, tens of billions of dollars (at least), and is most likely to happen in 2026-2027.
A month later, he clarified:
Possibly by 2026 or 2027 (and almost certainly no later than 2030), the capabilities of AI systems will be best thought of as akin to an entirely new state populated by highly intelligent people appearing on the global stage—a ‘country of geniuses in a datacenter’—with the profound economic, societal, and security implications that would bring.
At the other end, a good example of long timelines is Ege Erdil, Co-founder of Epoch AI, whose median time for the ‘full automation of remote work’ is 2045 — 20 years away.
While experts continue to disagree on when AI will start having transformative impacts, they are clearly not stubbornly ignoring the evidence. For as Helen Toner explained in her great essay: ‘Long’ timelines to advanced AI have gotten crazy short. Before ChatGPT, short timelines used to mean something like ‘10 to 20 years, so since it could take a long time to prepare, we should start now’. Long timelines used to mean ‘there was no sign AGI will happen in the next 30 years, if it happened this century at all, so it is premature to do any work related to controlling advanced AI’. But now we see short timelines like Dario Amodei’s with genius level AI ‘almost certain’ to happen within the next 5 years, and many staunch proponents of long timelines are now saying we’ll reach human-level in just 10 or 20 years.
Here’s a nice graph 80,000 Hours put together of how the average forecasted time until AGI on the Metaculus prediction site has shortened from about 50 years to about 5 years in just a 5-year window:
So everyone is updating on the evidence and shortening their timelines, yet substantial disagreement remains.
This is often framed as a debate: that we should be trying to assess who is right — whether timelines really are short or long (or medium). People pick winners, affiliate with one side or the other, and rub it in whenever the latest evidence favours their preferred camp.
My central claim today is that for most of us, that is the wrong frame. You should have neither short timelines nor long timelines — but broad timelines. That is:
The correct epistemic response to the lasting expert disagreement is to have a broad distribution over AI timelines.
First, there is too much disagreement among very smart and informed people for it to be reasonable to have a narrow range of possible years. You would need to ascribe very little chance to some of your epistemic peers seeing things more clearly than you do, when that actually happens half the time. Moreover, a lot of these people are coming from different fields, bearing diverse insights, evidence, and time-tested heuristics that no single individual is in a good position to judge.
And second, many of these people themselves have a broad distribution over AI timelines. For example, take Daniel Kokotajlo. He is one of the authors of AI 2027 and is known as a leading figure in the short timelines camp. A few years back, his median date for AI systems “able to replace 99% of current fully remote jobs” was 2027, hence the name of the scenario. Though his timelines have lengthened a little and by the time they were writing it, 2027 had become more of an illustrative early scenario rather than his point where it was 50% likely to have arrived.
Kokotajlo has done a great job of being extremely transparent about his timelines, showing his predictions (along with their uncertainty) for a variety of different levels of powerful AI. Here is his current probability distribution for when we will have an AI system that is “At least as good as top human experts at virtually all cognitive tasks”:
His distribution has its peak (the mode) in 2028, but because the distribution is heavily skewed towards the right, there is only a 27% chance of it happening by that point. His median year is 2030. And his 80% interval (from the 10th to 90th centile) is from 2027 to some point after 2050.
This is a broad distribution. I think someone’s 80% interval is a decent way of expressing the range of times they think are credible. Here Kokotajlo is saying that it will likely happen between 1 and 25 years from now, but that there is a 1 in 5 chance that it doesn’t even fall into that wide range.
He’s not the only one with such a broad distribution. Here are the forecasts of Daniel Kokotajlo, Ajeya Cotra, and Ege Erdil from 2023, forecasting: “In what year would AI systems be able to replace 99% of current fully remote jobs?”:
Note that all three have the same kind of shape, just stretched differently. And despite their very different medians they actually have a lot of overlap (which this transparent shading brings out). This shows both that each expert has a broad distribution and that the expert community on the whole has an even broader one. Indeed, I think you could do a lot worse than just taking a mixture model of these three experts’ views. Interestingly, since 2023, Kokotajlo’s distribution has shifted to the right and Erdil’s to the left.
Here’s an illustrative distribution for AGI timelines used by Ben Todd of 80,000 Hours:
Dwarkesh Patel reproduced it in his post about AI timelines, saying that it pretty much represented his own uncertainty, giving his median date of 2032 for AI that “learns on the job as easily, organically, seamlessly, and quickly as a human, for any white-collar work.”
Here is Metaculus’s current community estimate for when AGI will be developed. Synthesizing the community’s collective uncertainty, it is very broad and has this same characteristic shape:
Here is Epoch AI’s summary of leading estimates of AI timelines from 2023:
These look a bit different as they are represented as cumulative probabilities of reaching transformative AI by a given time. But they are all very broad. Take a look at the range of years between when they cross 10% to when they cross 90%. Every single one has an 80%-interval at least 50 years wide.
What about researchers working on AI capabilities? Grace et al surveyed thousands of AI researchers who were presenting at their top academic conferences. They surveyed the researchers in 2022 (blue) and 2023 (red) about when “unaided machines can accomplish every task better and more cheaply than human workers”:
You can see the wild variation in individual forecasts (the thin lines) and that the timelines became about 30% shorter in a single year. But vast uncertainty remains. The aggregate community forecasts (the thick lines) have 80% intervals ranging from years to centuries.
I think everyone should have a distribution that is roughly this shape. Here’s mine:
It is for transformative AI, loosely defined as AI that would be powerful enough to take over the world were it misaligned, and which is doubling the rate of scientific and technological progress. It’s a similar shape to Kokotajlo’s, but broader, with a median of 2038 and an 80% interval ranging from 3 years to 100 years.
Let’s return to where we started, with Daniel Kokotajlo’s distribution for AI that is “At least as good as top human experts at virtually all cognitive tasks”:
While we often express our timelines as single numbers (such as the mode or the median), I don’t think that’s a helpful approach here. Look at that graph. What number sums it up? Its only real feature is the peak, but Kokotajlo is saying it is unlikely to happen by then (just a 27% chance). The median is often a better number to give, but here it is at a relatively undistinguished point on the graph (in 4 years’ time) and saying ‘4 years’ would obscure his point that he thinks there is a 10% chance it is within 1 year and a 10% chance it is beyond 25 years.
I think that if he talked through what he actually means by this distribution with a smart policy maker, they would finally get it and say:
Oh, so you are saying you have no idea when it will happen — it could be next year, or it could be 6 presidential terms from now. And you’re saying there is a 1 in 5 chance it isn’t even in that range.
I think that’s actually a pretty good summary, and it would sum up my own distribution as well. While ‘no idea when it will happen’ is underselling the information contained in this distribution, it is a much better summary than ‘4 years’ which would be understood by almost everyone as something like ‘between 3 and 5 years’. While academics might hope people interpret a named year as the median time, most people interpret it as the moment they are allowed to start complaining the predicted event hasn’t happened yet.
Indeed, these distributions are so hard to sum up with a single number, that I think a substantial amount of disagreement on timelines stems from people describing different parts of the same elephant. For example, both AI boosters and those concerned with existential risk talk a lot about short timelines because ‘we could see the world transformed in just a few years’ time’. It isn’t that they think we will see that, but that it is big if true, and has a decent chance of being true. In contrast, more conservative voices tend to focus on later years saying ‘it is more likely that it will take 10 to 20 years, than that it will take just a few’ (focusing on straight probability without weighting by importance or leverage).
Both of these can be true at the same time. Both are true on my own distribution.
A particular danger in communicating timelines with a single number is that it raises the chance that this named year will come and go without incident, and the people who mentioned it (or the wider community they are part of) will be written off as having a false or discredited view. I think we’re going to see some of this come 2027 due to the vast number of people who heard about that scenario, combined with the fact that so many media outlets reported it as a sharp prediction, rather than as it was intended: an important illustrative scenario.
As well as being bad for communication, compressing one’s uncertainty into a single number would be very bad for your own planning.
For example Kokotajlo’s distribution implies a 28% chance transformative AI will happen during the current presidential term, a 35% chance it will happen in the next term, a 13% chance it will be the one after that, with 24% left over spread among ever more distant terms:
These are very different scenarios and it would clearly be a mistake to just act as if the second one were correct since it is the most likely. That would eliminate the possibility of hedging against transformative AI coming soon, and of taking advantage of worlds where it comes late.
Rather than attempting to adjudicate which length of timelines is correct, I think we should be taking the frame of how to act (or plan) under deeply uncertain timelines.
That is, we should be treating this as an exercise in rational decision-making under uncertainty — in a situation where the stakes are high and the uncertainty is vast.
Let’s unpack some of the implications of this frame.
We’ll start with two mistakes that are all too common in the policy world.
First, uncertainty about AI timelines isn’t an excuse to just believe whichever timeline you want, so long as it is within the credible range. Sadly, I think many government ministers are likely to take this approach if an expert explains this broad uncertainty to them. While they would be right that the evidence isn’t sufficient to disprove their preferred timeline, it would be irresponsible of them to not allow for other credible possibilities. That would be like a mayor hearing there is a 20% chance the volcano next to their town erupts next year and feeling that they can continue to act as if it won’t, since it not erupting is also found credible by the experts. Uncertainty isn’t an excuse to assume a plausible outcome of your choice will occur, it is more that rationality requires you to respect every plausible outcome.
Second, we can’t just wait until the uncertainty is resolved. Sometimes that works, but here we know the uncertainty is very unlikely to be resolved until the events are upon us. At that stage it will be too late to enact all but the most knee-jerk responses. So feeling that the cloud of uncertainty gives you permission to delay acting is tantamount to committing to choose one of the bluntest and least effective options available.
Instead, we are going to need to act under uncertainty, taking into account the full range of credible possibilities.
How can we do that?
A natural and important idea is that of hedging against transformative AI coming soon —while we are least prepared. We could do that by shifting our portfolio of activities (or your individual contribution to humanity’s portfolio) to focus somewhat more on short timelines than the raw probabilities would warrant.
This makes a lot of sense. I strongly recommend governments, civil society, and academics do more to hedge against transformative AI coming early.
Though when it comes to the communities of professionals already working on helping the AI transition go well, I think they are already hedging strongly against early transformative AI. Indeed, there is even a risk that they are going beyond mere hedging, and are actively betting on it coming early. I’m not sure, as it is hard to know the full portfolio of work.
One certainly sees many more pleas for work aimed at very short timelines than for long timelines. But there are also strong reasons to consider long timelines in our planning, and ways in which work aimed at long timelines can also be extremely high leverage.
Let’s look at two key things that happen when timelines are longer.
In longer timelines, AI arrives in a world that doesn’t look like today. The longer it is until transformative AI appears, the more different the world will be at that key moment.
As a baseline, suppose it arrives soon, in 2028. Things will definitely be different to today, but we’d expect many of the broad brushstrokes to be similar. We would likely have the same US president, the same major players, the same main technologies. If transformative AI arrived within just two years, I’d bet it was something like the AI 2027 story where a lab recklessly got recursive self-improvement going.
Now suppose transformative AI arrives in 2035. That is not this presidential term or even the next one, but the one after that. Who knows who’d be in power, or what state the US would be in. The nine years would likely have seen major changes in the core technologies of AI (9 years before now there were no LLMs or transformers). We could well have different leading AI companies, perhaps as a result of a bubble having burst and taken out the overextended first-movers.
By 2035, export controls may well have backfired, helping China get ahead on chips by incentivising them to build out their own chip industry and giving them 13 years to get good at it. This was a key dynamic the White House considered while drafting the export controls, but they were focused on shorter timelines… By 2035, China may have also invaded Taiwan, depriving the West of their biggest source of chips.
By 2035, there may be double-digit unemployment from increasingly powerful AI systems and public sentiment about AI could be very strong. The Overton window for AI regulation will be in a very different place.
As may be the geopolitical order. The last nine years has seen the invasion of Ukraine, the increasing isolation of the US and a global pandemic. Another nine years could see a similar amount of change.
And if we haven’t played our cards right, those of us working on avoiding catastrophic risks from AI may have also lost a lot of power, with our ideas about AI risk being seen as discredited since so many years have passed without the truly transformative effects we were talking about.
In short, the longer the timelines the more different things will be — both in some systematic, predictable ways, and just from random diffusion and chaos. So taking longer timelines seriously means:
There are many kinds of things people can work on that can pay off handsomely, but only after a number of years. Things like:
If you just consider your impact during the next three years, most of these will be beaten by other shorter-term options. But as the years climb, longer-term options can have very high value. They aren’t always best, but for the right people or the right opportunities, they can be extremely impactful.
When I was a grad student, I realised how much good I could achieve if I donated much of my income over my career to help those in the poorest countries. And the more I thought about it, the more I thought I should start something — an organisation — to help other people to do this too. So Will MacAskill and I launched Giving What We Can in 2009. 17 years later, more than 10,000 people have joined us, having thousands of times as much impact as if I’d carried on alone.
This kind of compounding growth is one of the major ways that longer term projects can have very large multipliers, giving us a very big boost to our impact if timelines are in fact long.
Starting new fields can be similar. When I first met Allan Dafoe 10 years ago, I didn’t know what he was talking about when he spoke of ‘AI governance’ — a new field he was trying to found. Now it is a burgeoning field, with hundreds of practitioners, who are in high demand from many different governments.
When I started writing The Precipice, I wasn’t sure I should, because I thought AGI might just be too close. But as it turns out, there was time to write it and for it to have a real impact. I’m really glad I did, as I meet so many amazing people working on the biggest risks who tell me it was reading The Precipice that inspired them to do so. I think it is one of the best things I’ve done.
After it came out, I used to think that there just wasn’t enough time to write a further book — that we were really too close to the critical moment. We might be, but I think I was mistaken about the strength of this argument. The time horizon for a book to have real impact is about 5 years (time to plan the book, win a book deal, write the book, wait for publishers to publish it, then wait a year or more before it has sufficient impact in the world).
But I only think there is about a 1 in 5 chance of transformative AI coming in the next 5 years. So while a book may come out too late, that is only a 1 in 5 chance, leaving a book project with 80% as much expected value as I’d have naively calculated. So while there is a 1 in 5 chance I’d be kicking myself, on my views about AI timelines there isn’t actually that much of a haircut in expected value due to the chance it is too late.
That said, the chance of transformative AI arriving before your work pays off is only one factor affecting whether you should do work aiming at short or long timelines. Another is that AI safety and governance are likely to be more neglected now than they will be later. This creates an extra multiplier for the value of direct work in these areas now, and in some cases is a larger effect than the chance your work comes to fruition after transformative AI.
Overall, I think that longer term projects do get down-weighted by these considerations, but their advantages sometimes outweigh that — especially if they are shooting for a very big payoff. I’d guess that if someone looked at their options and thought the best option was one that took 5 to 10 years to pay off, then about half the time it would remain their best option even after taking AI timelines into consideration. After all, it is not uncommon for your best option to be several times better than your second best.
So I think the community of people working on transformative AI are likely underrating types of work that need five or more years in order to pay off. The ideal portfolio of activities aimed at making the AI transition go well should include a number of things that really help us succeed in worlds where we get longer to try.
But I want to stress that none of this implies we can slack off.
We’re in a race against AI timelines. It is just that we don’t know if that race is a sprint or a marathon. In either case, time is of the essence.
We have seen that there is substantial disagreement and uncertainty about when AI will start having transformative impacts on the world. This is because there just isn’t enough evidence to pin it down. My claim is that for the purposes of planning we should adopt neither short nor long timelines, but broad timelines:
The correct epistemic response to the lasting expert disagreement is to have a broad distribution over AI timelines.
Given this deep uncertainty we need to act with epistemic humility. We have to take seriously the possibility it will come soon and hedge against that. But we also have to take seriously the possibility that it comes late and take advantage of the opportunities that would afford us. The world at large is doing too little of the former, but those of us who care most about making the AI transition go well might be doing too little of the latter.
We need to take more seriously the possibility that the world will look very different at that time, which should broaden our own Overton windows about what kinds of plans could succeed. And we shouldn’t be ruling out all actions which take a long time to pay off. Even if they wouldn’t help in short timelines worlds, some actions more than make up for this with substantial impacts if timelines are long.
Funders, career advisors, and movement builders should be thinking about this with regards to how we act as a community: to the shape of the whole portfolio of work aimed at effectively improving the world. And each of us should be reflecting on what this deeply uncertain timing means for planning our own contributions over the years to come.
I really enjoyed the version of this post that you shared at EAG SF. For posterity, my reaction after listening to the EAG talk was: "Wow, that talk was annoyingly helpful."
Annoying because: the core premise is so obvious, and yet I found spelling out the implications surprisingly clarifying! I think listening to that talk may have substantially accelerated my thinking on a key life decision I have been mulling on.
Thank you so much, @Toby_Ord :) I strongly upvoted.
Thanks Angelina!
Annoying because: the core premise is so obvious, and yet I found spelling out the implications surprisingly clarifying!
This is actually the story of almost everything I've ever done:
It's a byproduct of trying to find the most important, unarguably true, yet neglected things. I then work very hard on finding the deepest explanations until I find the ways of presenting each claim that make it effortless to see.
I like to think of it as working at the border of the trivial and the profound.
I really like that phrase, "working at the border of the trivial and the profound".
The Ross Summer Mathematics Program's motto "think deeply of simple things" also seems apt with respect to your work; I thought of it when reading your scaling series. You have a knack for finding fertile perspectives on ostensibly well-trodden topics and conveying them in luminously clear ways to nonexperts like myself, so thank you.
This reply made me smile :) Thank you for helping refine and spread so many simple, important ideas! I and many people I know owe so much to that work.
I think of AGI (and human-level intelligence) as the cloud, and superintelligence as being above the cloud. They are useful concepts, despite their vagueness. But they’re markedly less useful when you get close to them. [...]
For my purposes, I think the key threshold is when the system is capable enough to cause dramatic, civilisational changes. For example, the point where AI could take over from humanity if misaligned, or has made 50% of people permanently unemployable, or has doubled the global rate of technological progress. I focus on this threshold because I think it matters most for planning our strategies and careers.
I think the example milestones you mention differ significantly from one another, and each carries substantial vagueness that compounds rather than resolves the vagueness issues you raised earlier in the essay.
For example, I don't know how to operationalize the point where "AI could take over from humanity", and I suspect people will disagree for years about whether that threshold has been reached, much as they have debated for years whether we have already achieved AGI. Similarly, it is unclear what it means for 50% of people to be "permanently unemployable" as opposed to merely unemployed.
If your goal is to ground the debate about timelines in something measurable and uncontroversial, it is worth thinking more carefully about milestones that actually serve that purpose. Otherwise, time will pass and you will likely find that these milestones will become markedly less useful as we get close to them.
I agree that these are vague and could come apart from each other. But I don't see any crisp, verifiable definitions that I could replace them with and serve the same purpose. I'm interested in forecasting transformative AI for the main purpose of forecasting when one has to have one's AI-related impact by. e.g. by when do we need to have solved alignment (or to have paused AI development)?
If I instead used a verifiable definition here, such as the “In what year would AI systems be able to replace 99% of current fully remote jobs?” that I cite in the essay, then you have to do further forecasting of how that time relates to the key things that matter (such as the deadline on AI alignment). Also, for crisp concrete definitions, one tends to then get hung up on estimating exactly how hard the final 1% of current fully remote jobs are, because that is central to the prediction. For example, are there 1% of current fully remote jobs that we only let a human do, e.g. for reasons of legal responsibility or personal relationships? Maybe? But that isn't relevant to the central features we care about.
I'm sure my definition could be improved (the focus of my essay isn't on my prediction but on the wider points about everyone's timelines), but I hope this explains why being "measurable and uncontroversial" need not make for the best thing to forecast.
Great post. I want to focus on one particular claim because I think it's important.
Though when it comes to the communities of professionals already working on helping the AI transition go well, I think they are already hedging strongly against early transformative AI.
There is (relatively) too much work on trying to solve alignment, and not enough work on trying to lengthen timelines. The shorter timelines are, the less likely it is that we'll be able to solve alignment, which makes alignment work look less compelling—although that's not the only factor in the tradeoff (there are also tractability concerns).
My argument in We won't solve non-alignment problems by doing research could be rephrased as: a lot of people are doing research on non-alignment problems on the assumption that timelines are long, but this strategy won't work if timelines are short. I've seen approximately zero research reports on non-alignment problems ask the question of how they'll get solved under short timelines (AFAICT the best answer is to pause AI* but at the moment I'm actively thinking about if there are other answers). So again in this case I don't think people are doing a good job of hedging against short timelines.
So I agree with the thesis in OP (that we should have "broad timelines"), but I disagree that people are actually doing a good job of hedging against early transformative AI.
*That is, it's the best answer conditional on short timelines. I lean toward it being the best answer unconditionally, but it's less clear—some research agendas may have sufficiently high EV under long(er) timelines that they're the best-EV choice overall.
I found this super helpful thank you, probably the best thing I've read about AI timelines in the last year actually. So so well communicated with small words and minimal jargon thank you!
I know you're mainly taking about the best thinking approach here, but how does this translate to communication about AI timelines? Distributions make a lot of sense to me but are very hard for most people to think in. This wouldn't be useful to communicate with for most of my friends, unless I maybe had an hour and a large napkin... I wonder if there is a way to communicate in a "distributy" like way with people who just aren't statistically minded?
If some regular person asks me when i think the AI apocalypse is coming, what's a good way to communicate? I don't want to just guess a year for all the reasons you've stated, but a distribution won't be understood either. In the past I've said something like " I really don't know but it could well be between 2030 and 2040", but my impression has been this seems pathetically vague and unhelpful to most people. Any ideas on communicating AI timelines with integrity to non-statsy folks?
As a side note it seems strange that the guy who wrote the AI 2027 story's 50 percent point is at about 2031ish? Why wasn't the story then AI 2031?
Thanks Nick!
How to communicate this is a good question, and I don't yet know the best answer. I think admitting uncertainty is generally good — it is both honest and actually appreciated by many audiences. But there is still the question of how to do it. The scientific norm is usually to stay silent until the evidence for something (or some piece of the puzzle) is strong enough (e.g. reaching p = 0.05). I don't think that is the right norm here. We are in a very high stake situation and policy-makers need the partial evidence that we do have. But communicating it is hard.
I think your expression is pretty good, and could be made a little better. e.g. "Leading AI forecasters can't rule out it happening before 2030 and think it will probably happen before 2040."
Re AI 2027, there is a good explanation of how their views have changed here.
Thanks Toby interesting one on the communication. For policy makers I think that communcation style can work OK, less so with my friends haha.
I'm still confused by why they picked 2027 even in 2025. Back when they made it, Daniel's median forecast was 2028 and Eli's 2031. Surely you then pick 2029 or 2030 for your scenario? Picking the "most likely year for it to happen" still feels a bit disingenous to me.
I'm still confused by why they picked 2027 even in 2025. Back when they made it, Daniel's median forecast was 2028 and Eli's 2031. Surely you then pick 2029 or 2030 for your scenario? Picking the "most likely year for it to happen" still feels a bit disingenous to me.
I'm not sure why picking the mode feels disingenuous to you, it feels fine to me as long as it's between roughly 15th and 85th percentiles and you are transparent about it.
The causual history of why it was 2027 is that this was Daniel's median when we started writing it, and it would have been a lot of work to rewrite our near-final draft to make it 2028 after Daniel changed his view. The reason it was based off of Daniel's view and not other authors' is that AI 2027 was ultimately supposed to represent his view rather than amalgam that include others' views. Giving a single person final say seems better than design by committee. That said, we had few strong disagreements.
The other authors considered 2027 plausible enough and close enough to a modal scenario that they (including me) felt happy to help with the project.
Edit: Daniel discusses his perspective here
Edit 2: It probably would have been reasonable for me to push for the timelines to be in between our views rather than Daniel's. I didn't really consider it because I thought 2027 was plausible enough and Daniel was leading the project. I think I also gave some weight to Linch's point about it being important to communicate that things could get crazy very soon, but I'm not sure if this was cruxy. However, memetic fitness wasn't an (explici)t consideration.
Thanks that's helpful
From what Daniel said I thought his median was 2028 when he started to write it? But that's perhaps a bit nitpicky.
I think there might be a wider EA/Rationalist Comms issue here when communicating with the general public. Communicating projects like this isn't just about whether it "feels" fine - I think its important to think about how it might come accross and future implications. To the general public, this scenario even in 2030 still feels mega-soon and sci-fi. The problem is if we go past 2027 now, many people will say "those tech-bro idiots they're always wrong" and might miss the point o the thing
If anything I think here picking a more conservative, tail end of the timeline (2028-2030) would have been better, to keep it relevant for longer.
I agree not the biggest deal though.
I think they were optimizing for a combination of concreteness (so there's an exact story to point to, where the 2027 story is "things go roughly as they expected" whereas 2028 and 2031 were pricing in different types of individually unexpected delays[1]), and for memetic value.
Compare: My best estimate is that this project will take me 6 months. However, if you ask me to write it out step-by-step, it'd take me 4 months.The 6 months include buffers for various delays, some expected and some unexpected.
I think for project time estimations as part of a larger plan, the 6 month reply is more useful. But for someone following along on my thinking process, or a manager/collegue/direct report trying to help me optimize, the 4 month step-by-step report might be easier to follow along and/or more useful to critique or improve.
The concreteness is fine makes sense for sure
Isn't then somewhere between 2028 and 2031 really "things go roughly as expected" and 2027 is "things go faster than expected if every AI improvement rolls out without roadblocks?" I feel like if you're going to put something out there in the public sphere as a leader in AI, a bit of timeline conservatism might be prudent. Not the biggest deal though I suppose
"I feel like if you're going to put something out there in the public sphere as a leader in AI, a bit of timeline conservatism might be prudent."
I see and respect that position, but you can imagine someone saying the opposite: "I feel like if you're going to put something out there in the public sphere as a leader in AI, it's probably prudent to warn people of significant risks that happens much sooner than people expect, even if you think it's less than 50% likely to happen then."
Plausibly you can get away with reporting 3-5 numbers.
For 3 numbers, 25th percentile, median, 75th percentile. This is the approach ("interquartile range") used for reporting SAT acceptance ranges in the US. So we have at least a prior example of a widely reported figure that people don't think "normal people"/high-schoolers and their parents would have too much trouble understanding.
For 5 numbers, something like 5th percentile, 25th percentile, median, 75th percentile, 95th percentile.
3-5 numbers obviously harder to communicate than 1 number, and less precise than the full distribution. But hopefully it's clear and useful enough to be good here.
I'm not sure I can get away with that? I would say for over 90% of people 3 numbers would add even more confusion than 2. The SAT example is encouraging, although Americans make up a small proportion of my friends and acquaintances.
Hi Toby. Thanks for the relevant post.
For example Kokotajlo’s distribution implies a 28% chance transformative AI will happen during the current presidential term, a 35% chance it will happen in the next term, a 13% chance it will be the one after that, with 24% left over spread among ever more distant terms
There is even more uncertainty in AI Futures' artificial superintelligence (ASI) timelines. The difference between the 90th and 10th percentile is 168 years for Daniel Kokotajlo (2027 to 2195), and 137 years for Eli Lifland (2028 to 2165).
So helpful and clear!
There's so much great stuff in here. I particularly loved how you noted that uncertainty about timelines isn't an excuse to believe whatever timeline you want, nor to forgo the necessary preparations in government, etc. I so wish that "wise planning" was the mentality that so many of our key public and private stakeholders and leaders embraced!
Executive summary: Given persistent expert disagreement about AI timelines, the author argues that adopting a broad distribution over when transformative AI will arrive—rather than committing to short or long timelines—is the epistemically humble and strategically sound approach, with implications for how individuals and communities should plan their work.
Key points:
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.
This is truly an excellent article. I strongly agree that we need to maintain epistemological humility when predicting AGI occurrences, but I also understand that people always crave an accurate prediction, even with insufficient evidence; people are always uneasy when facing the unknown. However, I believe that even when the future state distribution is unknown, we are not without reasonable strategies for decision-making. I was inspired after reading Alexander Turner's "Optimal Policies Tend to Seek Power," which suggests that when the future reward function is randomly distributed, retaining nodes with more choices is the optimal solution. I think this has been very helpful in my own decision-making; even when the future environment is unknown, I believe choosing the node with more approximate branches is always a good approach.
I agree with many particular points in this post and the apparent thesis[1], but also think most people[2]should focus on short timelines (contrary to the apparent implication of the post). The reasons why are:
The counterargument I'm most sympathetic to is that (1) a high fraction of the work should be focused on "better futures" and (2) for better futures work, the leverage is higher in longer timelines. (I don't currently agree with either of (1) or (2), but I'm very uncertain.)
Assuming the thesis is "our probability distribution should span a wide range (including Daniel's distribution as an example of a wide range) and we should take this into account in our decision making. ↩︎
Or at least most of the quality weighted labor supply. ↩︎
I might have a small difference between these stated probabilities and my full all considered view including defering to others. To avoid deference cascades, I usually state probabilities somewhat closer to my non-deference view. (It's hard to fully disentangle deference because my views are based on talking to a wide range of different people.) Post deference my distribution is a bit wider with a correspondingly longer median. But I don't think this makes much difference either way and deference also pulls up my probability on very short timelines. ↩︎