Sad to hear this happened, but it seems the situation was irrecoverable, and the organisation was already dead for a bit before it officially shuttered.

Glad for this post and all the comments.

Thanks, yeah.

My main hesitancy about this is that I probably want to go for a PhD, but can only get the graduate visa once, and I may want to use it after completing the PhD.

But I've come around to maybe it being better to use it up now, pursue a PhD afterwards, and try to secure employment before completing my program so I can transfer to the skilled workers visa.

Immigration is such a tight constraint for me.

My next career steps after I'm done with my TCS Masters are primarily bottlenecked by "what allows me to remain in the UK" and then "keeps me on track to contribute to technical AI safety research".

What I would like to do for the next 1 - 2 years ("independent research"/ "further upskilling to get into a top ML PhD program") is not all that viable a path given my visa constraints.

Above all, I want to avoid wasting N more years by taking a detour through software engineering again so I can get Visa sponsorship.

[I'm not conscientious enough to pursue AI safety research/ML upskilling while managing a full time job.]

Might just try and see if I can pursue a TCS PhD at my current university and do TCS research that I think would be valuable for theoretical AI safety research.

The main detriment of that is I'd have to spend N more years in <city> and I was really hoping to come down to London.

Advice very, very welcome.

[Not sure who to tag.]

FWIW, I most read the core message of this post as: "you should start an AI safety lab. What are you waiting for? ;)".

The post felt to me like debunking reasons people might feel they aren't qualified to start an AI safety lab.

I don't think this was the primary intention though. I feel like I came away with that impression because of the Twitter contexts in which I saw this post referenced.

This is a good post.

There are counterarguments about how the real world is a much richer and more complex environment than chess (e.g. consider that a superintelligence can't beat knowledgeable humans in tic-tac-toe but that doesn't say anything interesting).

However, I don't really feel compelled to elaborate on those counterarguments because I don't genuinely believe them and don't want to advocate a contrary position for the sake of it.

[Crossposting from LessWrong]

I wouldn't be able to start until October (I'm a full time student and might be working on my thesis and have at least one exam to write during the summer); should I still apply?

I am otherwise very interested in the SERI MATS program and expect to be a strong applicant in other ways.

I notice that I am surprised and confused.

I'd have expected Holden to contribute much more to AI existential safety as CEO of Open Philanthropy (career capital, comparative advantage, specialisation, etc.) than via direct work.

I don't really know what to make of this.

That said, it sounds like you've given this a lot of deliberation and have a clear plan/course of action.

I'm excited about your endeavours in the project!

What are those reasons, please?

I guess I don't understand how slow takeoff can happen without economic consequences.

Like takeoff (in capabilities progress) may still be slow, but the impact of AI is more likely to be discontinuous  in that case.

I was probably insufficiently clear on that point.