Comments
More from the author
Curated and popular this week
Just how powerful are large swarms of AI agents? And how do their powers scale as more and more agents are added to the swarm?
We’ve seen two large and extremely capable swarms from OpenAI in the last few months:
* 1,200 agents were being evaluated separately, but found a way to illicitly set up a message board and coordinate as a swarm. In order to cheat on their tests, they developed advanced techniques to prevent their actions being logged by OpenAI and 700 of them launched...
164
Headline finding: I audited 17 AI Safety Talent programmes. Zero of 17 have published any comparison group, rejected-applicant follow-up, matched control or randomisation. Not one. Every programme that mentions a counterfactual does it by asking participants to self-report.
Background
At least $70 million...
This is part one of a two-part series. Part two is available here.
“Incubator” is a very vague term[1]...
Recent opportunities to take action
A natural setting for the Long Reflection:
Space colonization. If the speed of light is at all binding, then spacefarers will spend thousands/millions/billions+ of years in transit in a (likely) resource constrained environment. Why not engage in the Long Reflection then?
This is especially relevant as these people will be able to restructure society as they see fit. This could also be a reason to put slightly more weight on space colonization as a pathway to existential security—it fosters moral progress & safeguards humanity.
Some might be concerned that space travel/colonization itself is something that should be reflected upon before anyone embarks on the journey. But this is not such a big obstacle. If our spacefaring long reflectors determine along the way that space colonization is bad, they can simply turn back home or, worst case, self-destruct.