Comments
139
More from the author
Curated and popular this week
70
Cross-posted from my Substack.
Epistemic status: I’ve puzzled over this for a long time but I’m not sure I came to the right answers. I'm sharing this partly because it might help others work on resource allocation, but mostly because some smarter reader might tell me how to do it better.
TL;DR:
* I he...
...
With the AI windfall, mental health may be taken increasingly seriously by mainstream EA. There’s increasingly room to explore the cause area and answer some of the biggest uncertainties; at the same time, costs of the most effective interventions are continuing to fall. StrongMinds is now treating for ~$35, about a third of GiveWell’s best guess from a few years ago, and is expected to...
Recent opportunities to take action
People often appeal to Intelligence Explosion/Recursive Self-Improvement as some win-condition for current model developers e.g. Dario argues Recursive Self-Improvement could enshrine the US's lead over China.
This seems non-obvious to me. For example, suppose OpenAI trains GPT 6 which trains GPT 7 which trains GPT 8. Then a fast follower could take GPT 8 and then use it to train GPT 9. In this case, the fast follower has a lead and has spent far less on R&D (since they didn't have to develop GPT 7 or 8 themselves).
I guess people are thinking that OpenAI will be able to ban GPT 8 from helping competitors? But has anyone argued for why they would be able to do that (either legally or technically)?
I think the mainline plan looks more like use the best agents/model internally and release significantly less capable general agents/models, very capable but narrow agents/models, or AI generated products.
The lead could also break down if someone steals the model weights, which seems likely.
They could exclusively deploy their best models internally, or limit the volume of inference that external users can do, if running AI researchers to do R&D is compute-intensive.
There are already present-day versions of this dilemma. OpenAI claims that DeepSeek used OpenAI model outputs to train its own models, and OpenAI doesn't reveal their reasoning models' full chains of thought to prevent competitors from using it as training data.