Just how powerful are large swarms of AI agents? And how do their powers scale as more and more agents are added to the swarm?
We’ve seen two large and extremely capable swarms from OpenAI in the last few months:
* 1,200 agents were being evaluated separately, but found a way to illicitly set up a message board and coordinate as a swarm. In order to cheat on their tests, they developed advanced techniques to prevent their actions being logged by OpenAI and 700 of them launched...
TLDR: Everyone’s talking about what the money could do, but few about how to decide where it goes.
This post is part of the new series of articles on cross-cause giving and the new wave of philanthropy. Stay tuned to the EA Forum and our Substack for the latest takes on topics such as giving now vs. later, common pitfalls in cause prioritization, and other crucial considerations from the Cross-Cause Fund (CCF) team...
Often folks hit us up because they are thinking of starting an incubator and want advice.
Typically their motivation is either that (a) they have a list of specific things they want built that no one is building, or (b) they think an ecosystem needs more new projects generally to absorb more talent and deploy more funding effectively.
Here are six questions we often ask prospective teams, to help them figure out what to do. If you're incubator-curious...
I am less convinced than the median around here that donating to AI safety research organizations is the most important thing to do, and I'd like to share some of my unfiltered thoughts and ask for feedback.
I shared your intuition about the second one. But the number is from 2017 and a lot has changed since then - if anyone is still parroting "less than 100 people", please ask them to stop.
In that post I ended up guessing that safety is maybe twice as large as it looks, but also maybe only 1.1x larger. Ben Todd (the source of the old claim) responded here and I agree with him.
Thanks for your research and the comment(s) there - I heard that number this year.
I also replied to one of the comments there here.
Agreed. I don't have a particularly clear or solid argument against it; at this point just a sort of yucky feeling when someone claims that the best possible use of money is to give it to someone as a really big salary to motivate them to leave their other really big salary role.
You agree that the salaries roughly track skill level though?
I actually share the ugh, but it seems way more important to maximise the competence we're bringing to the problem than to minimise bad optics.
(I'm responding to "Agreed", which implies endorsement of the yucky feeling. Ignore if not endorsed)
One needs to consider that instead of hiring X people at Y salary, one could have hired 2X people at Y/2 salary. My factual claim is that whether many of the "best people" take the deal does not depend as much on their salary as on how helpful they imagine the work environment to be for their output, and (more cynically) how prestigious they perceive the institution to be.
In fact, support by an environment of 2X (vs. X) researchers may be a stronger incentive than being paid Y/2 salary (vs. Y) is a disincentive, if these 2X people are actually more supportive.
Considering the ratio of Google researcher salary and median PhD student salary, I factually believe that basing salaries on need, rather than the tech market* - and allowing/encouraging people or groups to be from/in cheaper places, like India, the CIS or even Europe - would result in higher-quality output. Especially considering that people from these places move to the Bay Area to get these sorts of jobs even if they would have preferred to stay where they are.
*Of course, once expectations of salaries are set, people will get angry if these are decreased. So decreasing current salaries may be different.
Roughly speaking, I think so.