Mercial applications are now open, for anybody looking to build a project in AI safety!
Do models think stealing their own weights is good?
How likely is a war over Taiwan?
Does publishing safety research teach models to evade it?

Mercial is a remote, part-time research fellowship: one month, a facilitator, and a real problem in technical or conceptual AI safety. There are 11 streams this round, covering agent collusion, interpretability, compute governance, alignment evals and more. It’s made to build the track record you need to get into further AI safety programmes.

Fellowship runs 1 November – 1 December.
Streams and applications: https://mercial.uk/fellowship
Applications close 28 October.

I think most AI safety bootcamps could be improved significantly by shifting focus off from coding. None of the researchers I know code, and I think the counterfactual activity of reading / thinking high-level about concepts is better for building context and conceptual thinking. As well as this, the technical AI safety pipeline serves more than just technical research roles (people transition into startups, grantmaking, other non-technical roles), and I think context building serves all of these roles better than coding. I think a default for a lot of bootcamps is to follow the ARENA curriculum, which is in parts outdated or marginally bad (eg I think learning about the specifics of how SAEs work, instead of the huggingface incident is clearly not optimal).