Headline finding: I audited 17 AI Safety Talent programmes. Zero of 17 have published any comparison group, rejected-applicant follow-up, matched control or randomisation. Not one. Every programme that mentions a counterfactual does it by asking participants to self-report.
Background
At least $70 million...
Just how powerful are large swarms of AI agents? And how do their powers scale as more and more agents are added to the swarm?
We’ve seen two large and extremely capable swarms from OpenAI in the last few months:
* 1,200 agents were being evaluated separately, but found a way to illicitly set up a message board and coordinate as a swarm. In order to cheat on their tests, they developed advanced techniques to prevent their actions being logged by OpenAI and 700 of them launched...
A year ago I posted that YCombinator (YC) companies didn’t seem to be growing faster since the release of ChatGPT in 2022. I reran that experiment and found that 2023+ YC companies are arguably growing a bit faster than pre-2023 companies (including AfterQuery, ...
I'm having thoughts about bottlenecks on improving institutional decision-making based on some of the conversations I had yesterday at EAG: Reconnect. Due to my experience, I'm mostly oriented towards governmental institutions, and one of the barriers there is the attitudes of constituents/voters.
To oversimplify, there are 2 dimensions to this: incorrect beliefs about what effects policies will have on the world, and low empathy/narrow circles of moral concern.
As an example of what I mean, take immigration policy. Voters incorrectly believe that higher immigration results in lower wages for residents, but they also don't care as much as they could/should about improving the well-being of strangers who currently live in another country.
It seems to me, even aside from the cause area of improving institutional decision-making, that moral circle expansion across the broader population is one of the interventions that I feel most confident about leading to a brighter long-term future.
If I were designing a research agenda around this goal, I might structure it this way:
What are examples of times and places that moral circle expansion occurred? I'm currently pretty confident that widespread moral circle expansion has occurred and would want to double-check that assumption by studying historical trends in ethics by e.g. talking to sociologists and historians.
Root cause analysis on how and why moral expansion occurred in the past.
What currently correlates with different moral circle sizes across populations? Initial intuition here is that "scarcity mindset" drives small circles of moral concern. Is this accurate? What factors correlate with wider moral concern? Religiosity? Median income? GINI coefficient?
Figure out how I would design a study or studies to find out if increases/decreases in [religiosity/scarcity mindset/median income/etc.] cause moral circle expansion, and how persistent (sticky) that change is likely to be.
Find out if such studies have already been done, or implement them if they have not been done.
Are there more direct interventions that lead to broader (persistent) moral circle expansion? E.g. consumption of psychedelic substances (and destigmatization/decriminalization thereof)
If the prior avenues of study are fruitful, search for the most tractable and cost-effective interventions for moral circle expansion.
TBH, the above feels pretty trite and obvious to me now that I've typed it all out, so I'm sure other people have already written about this somewhere; does anybody have ideas for where I could read others' thoughts along these lines?
A classic post from Gwern on the subject: