Bio

Non-EA interests include chess and TikTok (@benthamite). We are probably hiring: https://metr.org/hiring 

How others can help me

Feedback always appreciated; feel free to email/DM me or use this link if you prefer to be anonymous.

Sequences
3

AI Pause Debate Week
EA Hiring
EA Retention

Comments
1185

Topic contributions
6

Thanks for the article! I think if your definition of "long term" is "10 years," then EAAs actually do often think on this time horizon or longer, but maybe don't do so in the way you think is best. I think approximately all of the conversations about corporate commitments or government policy change that I have been involved in have operated on at least that timeline (sadly, this is how slowly these areas move). 

For example, you can see this CEA where @saulius projects out the impacts of broiler campaigns a cool 200 years, and links to estimates from ACE and AIM which use a constant discount rate.

Ah yeah good point, I updated the text.

I'm excited about this series!

I would be curious what your take is on this blog post from OpenAI, particularly these two graphs:

Investment in compute powers leading-edge research and step-change gains in model capability. Stronger models unlock better products and broader adoption of the OpenAI platform. Adoption drives revenue, and revenue funds the next wave of compute and innovation. The cycle compounds.

While their argument is not very precise, I understand them to be saying something like, "Sure, it's true that the costs of both inference and training are increasing exponentially. However, the value delivered by these improvements is also increasing exponentially. So the economics check out." 

A naive interpretation of e.g. the METR graph would disagree: humans are modeled as having a constant hourly wage, so being able to do a task which is 2x as long is precisely 2x as valuable (and therefore can't offset a >2x increase in compute costs). But this seems like an implausible simplification.

Do we have any evidence on how the value of models changes with their capabilities?

EA Animal Welfare Fund almost as big as Coefficient Giving FAW now?

This job ad says they raised >$10M in 2025 and are targeting $20M in 2026. CG's public Farmed Animal Welfare 2025 grants are ~$35M.  

Is this right?

Cool to see the fund grow so much either way.

Hmm. I understood them to be saying they (semi-) voluntarily scaled back before phase 2 was complete, so we can't read that much into the fact that phases 2/3 (where the high quality journalism happens) didn't happen. Maybe I misunderstood?

My read of their phase one plan is that they were intending to get these pretty low quality tabloid stories as a springboard to getting higher quality stuff. Maybe that was a bad plan, but the fact that the bad tabloid articles were in fact bad tabloid articles doesn't seem to discredit that?

Is this just a statement that there is more low-hanging fruit in safety research? I.e., you can in some sense learn an equal amount from a two-minute rollout for both capabilities and safety, but capabilities researchers have already learned most of what was possible and safety researchers haven't exhausted everything yet. 

Or is this a stronger claim that safety work is inherently a more short-time horizon thing?

This is great. 

Thinking through my own area of Data Generation and why I would put it earlier than you, I wonder if there should be an additional factor like "Importance of quantity over quality."

For example, my sense is that having a bunch of mediocre ideas about how to improve pretraining is probably not very useful to capabilities researchers. But having a bunch of trajectories where an attacker inserts a back door is pretty useful to control research, even if these trajectories are all things that would only fool amateurs and not experts. And if you are doing some sample inefficient technique, you might benefit from having absolutely huge amounts of this relatively mediocre data - a thing even relatively mediocre models could help with.

Thanks Sarah! Is something written up about the CEA donation system? I'm surprised that that's a priority, but obviously know zero details.

Note that some of the founders have pledged to donate 80% of their equity. 

Load more