Comments
More from the author
Curated and popular this week
Just how powerful are large swarms of AI agents? And how do their powers scale as more and more agents are added to the swarm?
We’ve seen two large and extremely capable swarms from OpenAI in the last few months:
* 1,200 agents were being evaluated separately, but found a way to illicitly set up a message board and coordinate as a swarm. In order to cheat on their tests, they developed advanced techniques to prevent their actions being logged by OpenAI and 700 of them launched...
49
Often folks hit us up because they are thinking of starting an incubator and want advice.
Typically their motivation is either that (a) they have a list of specific things they want built that no one is building,  or (b) they think an ecosystem needs more new projects generally to absorb more talent and deploy more funding effectively.
Here are six questions we often ask prospective teams, to help them figure out what to do. If you're incubator-curious...
Summary:
First, I give several different angles on how I feel about reinforcement learning:
* Theoretical case: RL is a black-box source of agency — this should give us classic misalignment worries, especially compared to agency-via-scaffolding
* Recent incidents (huggingface etc) and more mundane forms of misaligned behaviour in personal use give me bad vibes about the direction-of-travel of recent AI progress
* I’m worried things might get worse:...
Recent opportunities to take action
Whenever I talk about Effective Altruism (EA) to someone new, I talk about EA-the-Movement and EA-the-Philosophy. EA-the-Movement draws a specific kind of person (quantitative, techy, philosophical) and has selected a few causes it has determined to be the most effective. EA-the-Philosophy is about asking whether our donations and volunteering are going to places that get the most bang for our buck and those questions can be applied to anything we care about.Â
It's a way of easing people into our way of thinking without insisting that they join our particular group or adopt our priorities. I find it's especially useful if the quantitative or strong recommendations from EA-the-Movement to be offputting, or if they have previous associations with the movement. I think it's worth making people who are doing good in some way more effective, even if it doesn't end up getting them to do what we'd consider the most good. Although if someone spends enough time thinking with the EA Philosophy, it might end up leading the straight back to the EA Movement.Â
So, for example, if you're talking with a feminist you can ask them where their donations or volunteering could do more or the most good, and they could think about that and come up with answers. And you could give suggestions, e.g. charities that focus on girls education in LMICs, or reproductive health for women, or maternal health charities.Â
If you're talking with MRAs, you could ask them what are the more effective ways to help men and boys, and they could research ways to help in cases of IPV or MGC, etc.
And the same goes for environmentalists, civilisation collapse doomers, vegetarians and vegans, liberals, conservatives, proponents of free and open source software, people suffering from specific physical and mental health disorders, and so on.Â
Do you think it's useful to talk with people online (e.g. on YouTube, X, other social media, forums) about EA-as-philosophy? Although I think that EA-as-philosophy still requires some quantitative analysis of data, research and ethics to find better ways to help, people can still simply ask each other about how they can better help people.Â