I am really grateful for your courage, and for making it easier for others to speak up about the mistakes senior people in our community are making. I think the world would benefit a ton from more courage from the EA community for many reasons. Thank you for setting such a positive example.
Taking uni organizing really seriously was upstream of MATS, EA Courses/Virtual Programs, and BlueDot (shoutout to Dewi) getting started among other things. IMO this work is extremely valuable and heavily under-prioritized in the community compared to research. Group organizing can be quite helpful for training communications skills, entrepreneurship, agency, grit, improved intuitions about theories of change, management, networking/providing value to other people, general organization/ability to get things done, and many other flexible skills that from personal experience can significantly increase your impact.
Conditioned on human extinction, do you expect intelligent life to re-evolve with levels of autonomy similar to what humanity has now (which seems quite important for assessing how bad human extinction would be on longtermist grounds)? I don't think it's likely.
Maybe the underlying crux (if your intuition differs) is what proportion of human extinction scenarios (not including non-extinction x-risk) involve intelligent/agentic AIs, and/or other conditions which would significantly limit the potential of new intelligent life even if it did re-emerge. My current low-resilience impression is probably 90+%.
And the above considerations and credences make how good the next intelligent species are vs. humans fairly inconsequential.
Thanks for the feedback, and I’m sorry for causing that unintended (but foreseeable) reaction. I edited the wording of the original take to address your feedback. My intention for writing this was to encourage others to figure things out independently, share our thinking, and listen to our guts - especially when we disagree with the aforementioned sources of deference about how to do the most good.
I think EAs have done a surprisingly good job at identifying crucial insights, and acting accordingly. EAs also seem unusually willing to explicitly acknowledge opportunity cost and trade-offs (which I often find the rest of the world frustratingly unwilling to do). These are definitely worth celebrating.
However, I think our track record at translating the above into actually improving the future is nowhere near our potential.
Since I experience a lot of guilt about not being a good enough person, the EA community has provided a lot of much-needed comfort to handle the daunting challenge of doing as much good as I can. It’s been scary to confront the possibility that the “adults in charge” don’t have the important things figured out about how to do the most good. Given how the last few years have unfolded, they don’t even seem to be doing a particularly good job. Of course, this is very understandable. FTX trauma is intense, and the world is incredibly complicated. I don’t think I’m doing a particularly good job either.
But it has been liberating to allow myself to actually think, trust my gut, and not rely on the EA community/funders/orgs to assess how much impact I’m having relative to my potential. I expect that with more independent thinking, ambition, and courage, our community will do much better at realizing our potential moving forward.
I really appreciated this post, and think there is a ton of room for more impact with more frequent and rigorous cross-cause prioritization work. Your post prompted me to finally write up a related quick take I've been meaning to share for a while (which I'll reproduce below), so thank you!
***
I've been feeling increasingly strongly over the last couple of years that EA organizations and individuals (myself very much included) could be allocating resources and doing prioritization much more effectively. That said, I think we're doing extremely well in relative terms, and greatly appreciate the community's willingness to engage in such difficult prioritization.
Reasons why I think we're not realizing our potential:
Not realizing our lack of clarity about how to most impactfully allocate resources (time, money, attention, etc). Relatedly, an implicit assumption that we're doing a pretty good job, and couldn't be doing significantly better. I speculate that FTX trauma has significantly reduced the community's ambition to an unwarranted degree.
Path dependence + insufficient intervention prioritization research (in terms of quality, quantity, and frequency). I thoughtthis post brought up good points about the relative lack of cross-cause prioritization research + dissemination in the EA community despite its importance.
Being insufficiently open-minded about which areas and interventions might warrant resources/attention, and unwarranted deference to EA canon, 80K, Open Phil, EA/rationality thought leaders, etc.
Poorly applying heuristics as a substitute to prioritization work. See this comment and discussion about the neglectedness heuristic causing us to miss out on impact. FWIW I believe this very strongly and think the community has missed out on a ton of impact because of this specific mistake, but I’m unable to write about specifics in detail publicly. Feel free to reach out if you’d like to discuss this in private.
Aversion to feeling uncertainty and confusion (likely exacerbated by stress about AGI timelines).
Attachment to feeling certainty, comfort and assurance about the ethical and epistemic justification of our past actions, thinking, and deference.
Being slow to re-orient to important, quickly-evolving technological and geopolitical developments, and being unresponsive to certain kinds of evidence (e.g. inside AI world - not taking into account important political developments, outside of AI world - not taking AGI into account).
Strong non-impact tracking incentives (e.g. strong social incentives to have certain beliefs, work at certain orgs, focus on certain topics), and weak incentives to figure out and act on what is actually most impactful. We don't hear future beings (or current beings for the most part) letting us know that we could be helping them much more effectively by taking one action over another. We do feel judgment from our friends/in group very saliently.
Lacking the self-confidence/agency/courage/hero-licensing/interest/time/etc to figure things out ourselves, and share what we believe (and what we're confused about) with others - especially when it diverges from the aforementioned common sources of deference.
This is a shame given how brilliant, dedicated, and experienced members of the community are, and how much insight people have to offer – both within the community, and to the broader world.
I'm collaborating on a research project exploring how to most effectively address concentration of power risks (which I think the community has been neglecting) to improve the LTF/mitigate x-risk, considering implications of AGI and potentially short timelines, and the current political landscape (mostly focused on the US, and to a lesser extent China). We're planning to collate, ideate, and prioritize among concrete interventions to work on and donate to, and compare their effectiveness against other longtermist/x-risk mitigation interventions. I'd be excited to collaborate with others interested in getting more clarity on how to best spend time, money, and other resources on longtermist grounds. Reach out (e.g. by EA Forum DM) if you're interested. :)
I would also love to see more individuals and orgs conduct, fund, and share more cross-cause prioritization analyses (especially in areas under-explored by the community) with discretion about when to share publicly vs. privately.
I've been feeling increasingly strongly over the last couple of years that EA organizations and individuals (myself very much included) could be allocating resources and doing prioritization much more effectively. (That said, I think we're doing extremely well in relative terms, and greatly appreciate the community's willingness to engage in such difficult prioritization.)
Reasons why I think we're not realizing our potential:
Not realizing our lack of clarity about how to most impactfully allocate resources (time, money, attention, etc). Relatedly, an implicit assumption that we're doing a pretty good job, and couldn't be doing significantly better. I speculate that FTX trauma has significantly reduced the community's ambition to an unwarranted degree.
Path dependence + insufficient intervention prioritization research (in terms of quality, quantity, and frequency). I thoughtthis post brought up good points about the relative lack of cross-cause prioritization research + dissemination in the EA community despite its importance.
Being insufficiently open-minded about which areas and interventions might warrant resources/attention, and unwarranted deference to EA canon, 80K, Open Phil, EA/rationality thought leaders, etc.
Poorly applying heuristics as a substitute to prioritization work. See this comment and discussion about the neglectedness heuristic causing us to miss out on impact. FWIW I believe this very strongly and think the community has missed out on a ton of impact because of this specific mistake, but I’m unable to write about specifics in detail publicly. Feel free to reach out if you’d like to discuss this in private.
Aversion to feeling uncertainty and confusion (likely exacerbated by stress about AGI timelines).
Attachment to feeling certainty, comfort and assurance about the ethical and epistemic justification of our past actions, thinking, and deference.
Being slow to re-orient to important, quickly-evolving technological and geopolitical developments, and being unresponsive to certain kinds of evidence (e.g. inside AI world - not taking into account important political developments, outside of AI world - not taking AGI into account).
Strong non-impact tracking incentives (e.g. strong social incentives to have certain beliefs, work at certain orgs, focus on certain topics), and weak incentives to figure out and act on what is actually most impactful. We don't hear future beings (or current beings for the most part) letting us know that we could be helping them much more effectively by taking one action over another. We do feel judgment from our friends/in group very saliently.
Lacking the self-confidence/agency/courage/hero-licensing/interest/time/etc to figure things out ourselves, and share what we believe (and what we're confused about) with others - especially when it diverges from the aforementioned common sources of deference.
This is a shame given how brilliant, dedicated, and experienced members of the community are, and how much insight people have to offer – both within the community, and to the broader world.
I'm collaborating on a research project exploring how to most effectively address concentration of power risks (which I think the community has been neglecting) to improve the LTF/mitigate x-risk, considering implications of AGI and potentially short timelines, and the current political landscape (mostly focused on the US, and to a lesser extent China). We're planning to collate, ideate, and prioritize among concrete interventions to work on and donate to, and compare their effectiveness against other longtermist/x-risk mitigation interventions. I'd be excited to collaborate with others interested in getting more clarity on how to best spend time, money, and other resources on longtermist grounds. Reach out (e.g. by EA Forum DM) if you're interested. :)
I would also love to see more individuals and orgs conduct, fund, and share more cross-cause prioritization analyses (especially in areas under-explored by the community) with discretion about when to share publicly vs. privately.
To add a bit of context in terms of on-the-ground community building, I've been working on EA and AI safety community building at MIT and Harvard for most of the last two years (including now), though I have been more focused on AI safety field-building. I've also been helping out with advising for university EA groups, workshops/retreats for uni group organizers (both EA and AI safety), and organized beginning-of-year residencies at a few universities to support beginning-of-year EA outreach in 2021 and 2022 along with other miscellaneous EA CB projects (e.g. working with the CEA events team last year).
I do agree though that my experience is pretty different from that of regional/city/national group organizers.
Thank you for all your encouragement over the past few years for students and newer community members to post on the forum, and for actually making it easier and less scary to do so. I definitely would not have felt anywhere near as comfortable getting started without your encouragement and post editing offers. I've replaced Facebook binging with EA Forum binging since I both enjoyed it so much and found it really valuable for my learning. You will be missed, and incredibly hard to replace. Thank you for all your hard work!
I am really grateful for your courage, and for making it easier for others to speak up about the mistakes senior people in our community are making. I think the world would benefit a ton from more courage from the EA community for many reasons. Thank you for setting such a positive example.
The argument sure is a string :P
Will's list from his recent post has good candidates too:
Taking uni organizing really seriously was upstream of MATS, EA Courses/Virtual Programs, and BlueDot (shoutout to Dewi) getting started among other things. IMO this work is extremely valuable and heavily under-prioritized in the community compared to research. Group organizing can be quite helpful for training communications skills, entrepreneurship, agency, grit, improved intuitions about theories of change, management, networking/providing value to other people, general organization/ability to get things done, and many other flexible skills that from personal experience can significantly increase your impact.
Conditioned on human extinction, do you expect intelligent life to re-evolve with levels of autonomy similar to what humanity has now (which seems quite important for assessing how bad human extinction would be on longtermist grounds)? I don't think it's likely.
Maybe the underlying crux (if your intuition differs) is what proportion of human extinction scenarios (not including non-extinction x-risk) involve intelligent/agentic AIs, and/or other conditions which would significantly limit the potential of new intelligent life even if it did re-emerge. My current low-resilience impression is probably 90+%.
And the above considerations and credences make how good the next intelligent species are vs. humans fairly inconsequential.
Thanks for the feedback, and I’m sorry for causing that unintended (but foreseeable) reaction. I edited the wording of the original take to address your feedback. My intention for writing this was to encourage others to figure things out independently, share our thinking, and listen to our guts - especially when we disagree with the aforementioned sources of deference about how to do the most good.
I think EAs have done a surprisingly good job at identifying crucial insights, and acting accordingly. EAs also seem unusually willing to explicitly acknowledge opportunity cost and trade-offs (which I often find the rest of the world frustratingly unwilling to do). These are definitely worth celebrating.
However, I think our track record at translating the above into actually improving the future is nowhere near our potential.
Since I experience a lot of guilt about not being a good enough person, the EA community has provided a lot of much-needed comfort to handle the daunting challenge of doing as much good as I can. It’s been scary to confront the possibility that the “adults in charge” don’t have the important things figured out about how to do the most good. Given how the last few years have unfolded, they don’t even seem to be doing a particularly good job. Of course, this is very understandable. FTX trauma is intense, and the world is incredibly complicated. I don’t think I’m doing a particularly good job either.
But it has been liberating to allow myself to actually think, trust my gut, and not rely on the EA community/funders/orgs to assess how much impact I’m having relative to my potential. I expect that with more independent thinking, ambition, and courage, our community will do much better at realizing our potential moving forward.
I really appreciated this post, and think there is a ton of room for more impact with more frequent and rigorous cross-cause prioritization work. Your post prompted me to finally write up a related quick take I've been meaning to share for a while (which I'll reproduce below), so thank you!
***
I've been feeling increasingly strongly over the last couple of years that EA organizations and individuals (myself very much included) could be allocating resources and doing prioritization much more effectively. That said, I think we're doing extremely well in relative terms, and greatly appreciate the community's willingness to engage in such difficult prioritization.
Reasons why I think we're not realizing our potential:
I'm collaborating on a research project exploring how to most effectively address concentration of power risks (which I think the community has been neglecting) to improve the LTF/mitigate x-risk, considering implications of AGI and potentially short timelines, and the current political landscape (mostly focused on the US, and to a lesser extent China). We're planning to collate, ideate, and prioritize among concrete interventions to work on and donate to, and compare their effectiveness against other longtermist/x-risk mitigation interventions. I'd be excited to collaborate with others interested in getting more clarity on how to best spend time, money, and other resources on longtermist grounds. Reach out (e.g. by EA Forum DM) if you're interested. :)
I would also love to see more individuals and orgs conduct, fund, and share more cross-cause prioritization analyses (especially in areas under-explored by the community) with discretion about when to share publicly vs. privately.
I've been feeling increasingly strongly over the last couple of years that EA organizations and individuals (myself very much included) could be allocating resources and doing prioritization much more effectively. (That said, I think we're doing extremely well in relative terms, and greatly appreciate the community's willingness to engage in such difficult prioritization.)
Reasons why I think we're not realizing our potential:
I'm collaborating on a research project exploring how to most effectively address concentration of power risks (which I think the community has been neglecting) to improve the LTF/mitigate x-risk, considering implications of AGI and potentially short timelines, and the current political landscape (mostly focused on the US, and to a lesser extent China). We're planning to collate, ideate, and prioritize among concrete interventions to work on and donate to, and compare their effectiveness against other longtermist/x-risk mitigation interventions. I'd be excited to collaborate with others interested in getting more clarity on how to best spend time, money, and other resources on longtermist grounds. Reach out (e.g. by EA Forum DM) if you're interested. :)
I would also love to see more individuals and orgs conduct, fund, and share more cross-cause prioritization analyses (especially in areas under-explored by the community) with discretion about when to share publicly vs. privately.
To add a bit of context in terms of on-the-ground community building, I've been working on EA and AI safety community building at MIT and Harvard for most of the last two years (including now), though I have been more focused on AI safety field-building. I've also been helping out with advising for university EA groups, workshops/retreats for uni group organizers (both EA and AI safety), and organized beginning-of-year residencies at a few universities to support beginning-of-year EA outreach in 2021 and 2022 along with other miscellaneous EA CB projects (e.g. working with the CEA events team last year).
I do agree though that my experience is pretty different from that of regional/city/national group organizers.
Thank you for all your encouragement over the past few years for students and newer community members to post on the forum, and for actually making it easier and less scary to do so. I definitely would not have felt anywhere near as comfortable getting started without your encouragement and post editing offers. I've replaced Facebook binging with EA Forum binging since I both enjoyed it so much and found it really valuable for my learning. You will be missed, and incredibly hard to replace. Thank you for all your hard work!