This is a nice document that has produced a lot of interesting discussion in the GDoc comments. I'm struck by how a handful of the ideas were already done or being done, suggesting that .impact isn't succeeding in its aim to be a venue for communicating and coordinating EA projects. I'd be interested to hear ideas for how to better do that, as I think communication and coordination is somewhat lacking.
I don't generally have the impression that when people have a new EA idea, they check if .impact has done or is doing it. I've seen many examples of duplication or near-duplication*.
Partly it's an advertising issue, partly I think that when .impact is marketed it's more 'If you want projects to work on, come to this' as opposed to 'if you have an idea for a project, talk to us'.
*To take the one that has come up most, I think at least 4 people from different groups have independently talked to me about a REG-for-finance-people concept.
I'm currently participating in the MIINT Impact Investing programme (www.themiint.org) at the Said Business School in Oxford, and trying to find opportunities to invest in high-impact/low-medium-return projects or companies. They need to be at a fairly early stage in their development and looking to raise $250k to $750k in the next round. We're cause agnostic and considering any opportunities to improve welfare for anything sentient.
If it exists, can you please point me in the direction of something like the above list, but for existing companies? I've not been able to find any info on EAventures; did it not get off the ground?
Just how powerful are large swarms of AI agents? And how do their powers scale as more and more agents are added to the swarm?
We’ve seen two large and extremely capable swarms from OpenAI in the last few months:
* 1,200 agents were being evaluated separately, but found a way to illicitly set up a message board and coordinate as a swarm. In order to cheat on their tests, they developed advanced techniques to prevent their actions being logged by OpenAI and 700 of them launched...
Often folks hit us up because they are thinking of starting an incubator and want advice.
Typically their motivation is either that (a) they have a list of specific things they want built that no one is building,  or (b) they think an ecosystem needs more new projects generally to absorb more talent and deploy more funding effectively.
Here are six questions we often ask prospective teams, to help them figure out what to do. If you're incubator-curious...
Summary:
First, I give several different angles on how I feel about reinforcement learning:
* Theoretical case: RL is a black-box source of agency — this should give us classic misalignment worries, especially compared to agency-via-scaffolding
* Recent incidents (huggingface etc) and more mundane forms of misaligned behaviour in personal use give me bad vibes about the direction-of-travel of recent AI progress
* I’m worried things might get worse:...
This is a nice document that has produced a lot of interesting discussion in the GDoc comments. I'm struck by how a handful of the ideas were already done or being done, suggesting that .impact isn't succeeding in its aim to be a venue for communicating and coordinating EA projects. I'd be interested to hear ideas for how to better do that, as I think communication and coordination is somewhat lacking.
I don't generally have the impression that when people have a new EA idea, they check if .impact has done or is doing it. I've seen many examples of duplication or near-duplication*.
Partly it's an advertising issue, partly I think that when .impact is marketed it's more 'If you want projects to work on, come to this' as opposed to 'if you have an idea for a project, talk to us'.
*To take the one that has come up most, I think at least 4 people from different groups have independently talked to me about a REG-for-finance-people concept.
That's a great point Tom. Is .impact widely advertised?
No, I think it's fairly poorly advertised. I plan to post about it here on the forum at some point (though it'd be great if someone beat me to it!) This poll suggested knowledge was middling: https://www.facebook.com/groups/effective.altruists/permalink/832289520160740/?qa_ref=qd
I've put this on the agenda to discuss in tomorrow's meeting, including asking if anyone would be up for writing that post.