Headline finding: I audited 17 AI Safety Talent programmes. Zero of 17 have published any comparison group, rejected-applicant follow-up, matched control or randomisation. Not one. Every programme that mentions a counterfactual does it by asking participants to self-report.
Background
At least $70 million...
In celebration of still being alive and fighting, we are giving away 1,000 Amazon e-books of “If Anyone Builds It, Everyone Dies”. Feel free to send a copy to yourself, a loved one, or a friend—we need all hands on deck.
Today marks exactly one year since If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All, by Eliezer Yudkowsky and Nate Soare...
Just how powerful are large swarms of AI agents? And how do their powers scale as more and more agents are added to the swarm?
We’ve seen two large and extremely capable swarms from OpenAI in the last few months:
* 1,200 agents were being evaluated separately, but found a way to illicitly set up a message board and coordinate as a swarm. In order to cheat on their tests, they developed advanced techniques to prevent their actions being logged by OpenAI and 700 of them launched...
I've been running into something I think of as "The Narrow Band Dilemma" where a moderate ethical position is fragile because it is in a battle on two fronts between a popular more pure ethical position and a popular unethical or a-moral position.
The first example is ethically produced meat / free-range farming, where a slightly more expensive product tries to find a market that loses patrons on both sides, either people who don't care about where their meat comes from, or care more about the expense than the ethics on the one hand, and the people who forgo meat altogether (vegans / vegetarians) on the other. As messaging successfully advertises the benefits of ethical meat production, it gains market share from one end of the spectrum (the "I care about ethics but can't justify the additional expense") but loses them on the other because the heightened awareness of animal cruelty drives ethical meat eaters away from eating meat altogether.
I noticed another example today in moderate Christianity, where it is flanked by fundamentalism and atheism. The more that Christians want to align their beliefs with modern secular ethics (moving away from fundamentalism), the more they are likely to leave the faith altogether.
Now, I'm not actually saying this is a problem, just a phenomenon I've noticed, I'm an atheist, so don't mind if more people come to think as I do, but I'd also like more Christians to be more moderate, but I can see it's difficult to build critical mass when they exist in a narrow band. I am in the narrow band when it comes to ethical meat, and I have seen how long it has taken for ethical meat products to become ubiquitous and accessible, and truly ethical real meat (lab grown) is still a way off. I imagine, if a much larger market had emerged (without the pressures of the narrow band), it's possible this could be available already.