Just how powerful are large swarms of AI agents? And how do their powers scale as more and more agents are added to the swarm?
We’ve seen two large and extremely capable swarms from OpenAI in the last few months:
* 1,200 agents were being evaluated separately, but found a way to illicitly set up a message board and coordinate as a swarm. In order to cheat on their tests, they developed advanced techniques to prevent their actions being logged by OpenAI and 700 of them launched...
Often folks hit us up because they are thinking of starting an incubator and want advice.
Typically their motivation is either that (a) they have a list of specific things they want built that no one is building, or (b) they think an ecosystem needs more new projects generally to absorb more talent and deploy more funding effectively.
Here are six questions we often ask prospective teams, to help them figure out what to do. If you're incubator-curious...
Summary:
First, I give several different angles on how I feel about reinforcement learning:
* Theoretical case: RL is a black-box source of agency — this should give us classic misalignment worries, especially compared to agency-via-scaffolding
* Recent incidents (huggingface etc) and more mundane forms of misaligned behaviour in personal use give me bad vibes about the direction-of-travel of recent AI progress
* I’m worried things might get worse:...
Thanks for letting us know about this great opportunity! While I'm waiting to be approved for the Facebook group, is there any way to find out how much money is going to be chasing the $2 million match? Since this is not just EAs and there appear to be hundreds of charities listed, it could easily be $100 million, so then do you think we will have something like 3 minutes or 3 seconds to do the donation before the $2 million match limit is reached?
This year I expect it to be more, though I'm not well-informed on how much more. Perhaps $10-$20MM is a reasonable expectation. https://en.wikipedia.org/wiki/Giving_Tuesday
Also, the match last year was for $500K instead of $2MM. From the same source:
Note that last year's matching campaign was also announced in advance.
So I think 3 minutes is overkill. While apriori I would be expecting people to take advantage of this such that $2MM in donations are made in the first ~3 minutes, I think that last year shows that this is unlikely to happen. I would be surprised if the $2MM match is reached in less than 30 minutes. I'll assign a 20% probability to that happening somewhat arbitrarily. And maybe a 5% chance to it being reached in less than 10 minutes. My median estimate would be around 9:30 EST (1.5 hours). And maybe a 20% chance that it takes more than 3 hours. Although I don't really know, so my suggestion is to donate ASAP. If you're donating more than just a small amount it's worth it even if it's inconvenient.
I intend to make all of my donations ASAP after 8:00 AM EST. (I am going to try to make 10 separate $1,000 donations before 8:10 AM EST).