a) The link to your post on defining alignment research is broken
b) "Governing with AI opens up this whole new expanse of different possibilities" - Agreed. This is part of the reason why my current research focus is on wise AI advisors.
c) Regarding malaria vaccines, I suspect it's because the people who focused on high-quality evidence preferred bet nets and folk who were interested in more speculative interventions were drawn more towards long-termism.

In retrospect, it seems that LLM's were initially successful because they allowed engineers to produce certain capabilities in a way that almost maximally leaned on crystallized knowledge and minimally leaned on fluid intelligence.

It appears that LLM's have continued to be successful because we've gradually been able to get them to rely more on fluid intelligence.

The AI Safety Fundamentals course has done a good job of building up the AI safety community and you might want to consider running something similar for moral alignment.

One advantage of developing on a broader moral alignment field is that you might be able to produce a course that would still appeal to folks who are skeptical of either the animal rights or AI sentience strands.

I can share a few comments on my thoughts here if this is something you'd consider pursuing.

(I also see possible intersections with my Wise AI advisor research direction).

Sorry to hear it didn't work out and thank you for your service.

For what it's worth, often it's valuable to take a step back rather than to just keep hitting your head against a wall. This can provide space to develop a better sense of perspective and why things went the way they did, whether you might have had a shot if you approached things in a different way or whether something else might be a better fit for you.

One thing I'd be much more excited about seeing rather than "quantifying post-training variables and their effects" (but which I'm not planning to pursue) would be to take an old model and then to try to map post-training enhancements discovered over time and see how the maximum elicitable capabilities change.

I'm worried that quantifying post-training variables directly has significant capabilities externalities and that there's no obvious limit to how far post-training can be pushed.

Comments:

1. I wouldn't really frame it as us wanting more folk to take AI safety courses as I think quality is more important than quantity. Instead, I would personally focus more on a) local knowledge about people's potential b) building up viable local scenes for people to grow/develop in.
2. Regarding marketing, I'd be much more interested in niche marketing targeting people who are top of the distribution in some way than just increasing numbers due to decreasing marginal returns.

Your model here seemed to be that you should only run a fellowship program if you already have a strong community, but running a fellowship program is a way to build up a community. It has a strong track record with university groups. Admittedly, it is slightly less suited to the country context, but it can still work either remotely or by running groups in the most major cities.

As an AI safety person who believes short timelines are very possible, I'm extremely glad to see this shift.

For those who are disappointed, I think it's worth mentioning that I just took a look at the Probably Good website and it seems much better than the last time I looked. I had previously been a bit reluctant to recommend it, but it now seems like a pretty good resource and I'm sure they'll be able to make it even better with more support.

Given that The 80,000 Hours Podcast is increasing its focus on AI, it's worth highlighting Asterisk Magazine as a good resource for exploring a broader set of EA-adjacent ideas.

Change in strategic direction?