It's the first chapter in a new guide about how to help make AI go well (aimed at new audiences).

I think it's generally important for people who want to help to understand the strategic picture.

Plus in my experience the thing most likely to make people take AI risk more seriously is believing that powerful AI might happen soon. 

I appreciate that talking about this could also wake more people up to AGI, but I expect the guide overall will proportionally boost the safety talent pool a lot more than the speeding up AI talent pool. 

(And long term I think it's also better to be open about my actual thinking rather than try to message control to that degree, and a big part of the case in favour in my mind is that it might happen soon.)

Yes I basically agree that's the biggest limiting factor at this point.

However, a better base model can improve agency via e.g. better perception (which is still weak).

And although reasoning models are good at science and math, they still make dumb mistakes reasoning about other domains, and very high reliability is needed for agents. So I expect better reasoning models also helps with agency quite a bit.

I feel subtweeted :p As far as I can tell, most of the wider world isn't aware of the arguments for shorter timelines, and my pieces are aimed at them, rather than people already in the bubble.

That said, I do think there was a significant shortening of timelines from 2022 to 2024, and many people in EA should reassess whether their plans still make sense in light of that (e.g. general EA movement building looks less attractive relative to direct AI work compared to before).

Beyond that, I agree people shouldn't be making month-to-month adjustments to their plans based on timelines, and should try to look for robust interventions.

I also agree many people should be on paths that build their leverage into the 2030s, even if there's a chance it's 'too late'. It's possible to get ~10x more leverage by investing in career capital / org building / movement building, and that can easily offset. I'll try to get this message across in the new 80k AI guide.

Also agree for strategy it's usually better to discuss specific capabilities and specific transformative effects you're concerned about, rather than 'AGI' in general. (I wrote about AGI because it's the most commonly used term outside of EA and was aiming to reach new people.)

Apparently there's a preprint showing Gemini 2.5 gets 20% on the Olympiad questions, which would be in line with the o3 result.

I wouldn't totally defer to them, but I wouldn't totally ignore them either. (And this is mostly besides the point since I'm overall I'm critical of using their forecasts and my argument doesn't rest on this.)

I only came across this paper in the last few days! (The post you link to is from 5th April; my article was first published 21st March.) 

I want to see more commentary on the paper before deciding what to do about it. My current understanding:

o3-mini seems to be a lot worse than o3 – it only got ~10% on Frontier Math, similar to o1. (Claude Sonnet 3.7 only gets ~3%.) 

So the results actually seem consistent with Frontier Math, except they didn't test o3, which is significantly ahead of other models.

The other factor seems to be that they evaluated the quality of the proofs rather than the ability to get a correct numerical answer.

I'm not sure data leakage is a big part of the difference.

Here we're also talking about capabilities rather than harm. If you want to find out how fast cars will be in 5 years, asking the auto industry seems like a reasonable move.

So, OpenAI is telling the truth when it says AGI will come soon and lying when it says AGI will not come soon?

I don't especially trust OpenAI's statements on either front.

The framing of the piece is "the companies are making these claims, let's dig into the evidence for ourselves" not "let's believe the companies".

(I think the companies are most worth listening to when it comes to specific capabilities that will arrive in the next 2-3 years.)

I agree those two statements don't obviously seem inconsistent, though independently it seems to me Dario probably has been too optimistic historically.

I discuss expert views here. I don't put much weight on the superforecaster estimates you mention at this point because they were made in 2022, before the dramatic shortening in timelines due to chatGPT (let alone reasoning models).

They also (i) made compute forecasts that were very wrong (ii) don't seem to know that much about AI (iii) were selected for expertise in forecasting near-term political events, which might not generalise very well to longer-term forecasting of a new technology.

I agree we should consider the forecast, but I think it's ultimately pretty weak evidence.

The AI experts survey also found a 25% chance of AI that "can do all tasks better than a human" by 2032. I don't know why they think it'll take so much longer to "automate all jobs" – it seems likely they're just not thinking about it very carefully (especially since they estimate ~50% of an intelligence explosion starting after AI can do "all tasks"); or it could be because they think there will be a bunch of jobs where people have a strong preference for a human to be in them (e.g. priest, artist), even if AI is technically better at everything. 

The AI experts have also been generally too pessimistic and e.g. in 2023 predicted that AI couldn't do simple Python programming until 2025, though it could probably already do that at the time. I expect their answers in the next survey will be shorter again. And they're also not experts in forecasting.