Comments
I’ve just released a post which offers significantly more empirical data on how FTX has impacted EA. FTX’s collapse seems to mark a clear and sizable deterioration across a variety of different EA metrics.
I’ve just released a post which offers significantly more empirical data on how FTX has impacted EA. FTX’s collapse seems to mark a clear and sizable deterioration across a variety of different EA metrics.
Fantastic study nice one
The headline finding here for me is the HUGE negative response within the EA community, especially the reduction in trust of leaders which will in turn have a roll on effect on our ability to reach out and grow the movement.
- 1 in 3 EAs "substantially losing trust" in the leadership is super high
- Half the community saying the crisis made them lose trust in our leadership orgs should be really concerning and prompt action (see below)
- A decrease of 0.5 - 1 out of 10 satisfaction on your scale is very large considering the scale
Perhaps this could prompt a little more action from the leadership and leadership orgs to regain our confidence in them and the movement in general.
1. More inspiration and fight from leadership - this might sound cheesy but I'd love to hear more positivity and fighting spirit from the more talismanic leaders of the movement. Things like videos and short messages would be great.
2. More activity on the forum from leadership and orgs (shouldn't be too hard)
3. Full transparency and clarity on the goals and plans of leadership orgs
Great study anyway and it lines up with my own feelings and intuitions as well.
The implicit assumption here I want to push back on is questioning whether we want the old trust systems that bordered on hero worship back. I do not long for the days where people commented on Will Macaskill's biceps and were incredibly deferential instead of questioning. I don't think it's good for the community health or the impact of EA.
I really like this point, but I also empathize with the desire for strong public voices focused on pitching EA ideas to the world. It does feel like it's easier to promote ideas when consistent sets of individuals are spokespeople and public influencers. I hope organizations and the community overall can promote a diverse set of individuals to that sort of platform without blind hero worship in the way you've described here!
Thanks I agree that hero worship isn't good - I suppose I never really experienced that myself so didn't see that negative side. I hope that in leadership we can have a diverse range of strong public voices which are also humble, involved at grass roots level and responsive to new ideas. I definitely don't advocate for any kind of "trust based" deference to leaders, but I think we still need good leaders all the same :).
I agree with you that the loss of trust in leaders really stands out. I think it's worth asking why that happened and what could have been done better. Presumably people will differ on this, but here's roughly how I would expect a crisis to be managed well:
What dented my trust was that I didn't and haven't observed 2 or 3 happening. When FTX blew up, various leaders seemed to stop communicating, citing legal reasons (which I don't understand and don't seem particularly plausible). I can't think of any major changes that have been proposed or enacted, even now (Cf Winston Churchill's "don't let a good crisis go to waste"). I think restoring trust from this position is tricky - because it now feels like it's been a long time since FTX without much happening. But what I would still like to see is those in central leadership positions is reflecting publicly on what's happened, what they've learnt, and sharing a vision for the future.
I see a couple of people have disagreed with this. Curiously to know why people disagree! Is this the wrong model of what crisis response looks like? Am I being too harsh about 2 and 3 not happening? Do I have the wrong model of what should happen to restore trust? Personally, I would love to feel that EA handled FTX really well.
I didn't disagree vote but did feel a bit like "getting people to share a vision for the future is kind of the whole point of EA Strategy Fortnight, no?"
Yes, glad the strategy fortnight is happening. But this is fully 6 months post-FTX. And I think it's fair to say there's been a lack of communication. IME people don't mind waiting so much, so long as they have been told what's going to happen.
This summary is a really helpful resource, thanks for sharing!
Have there been any studies (or plans to study) the effect of FTX on the perceptions of people in relevant overlapping social circles to EA that we would expect to recruit from via warm networks? For example, the Bay Area tech scene, ML researchers, academics in relevant disciplines at top 10 schools (etc.)?
My guess is that the impressions of EA on those groups (who are probably "EA-aware" but not "EA" - the 1% who know about EA) could be (much) more negative than either existing community members or the general public.
I am not aware of any plans to do that. I think it would be useful to get a sense of how those communities view EA; in particular I have heard various hypotheses about how AI safety projects should present themselves to the AI community, and would be good for that to be grounded in more data. Thanks for the suggestion!
I suspect that it will be several years before the full impact can be measured with a reasonable degree of certainty. I may be projecting, but I expect that the full impact will heavily depend on questions like:
(1) Were all appropriate investigations conducted and appropriate reports released, and what did they show about major EA actors and organizations?
(2) What concrete, legible actions did EA actors and organizations take in response to various problems identified by the FTX crisis?
(3) How did EA organizations that received FTX-linked money address their ethical and moral obligations to the victims of FTX fraud?
Are there plans to study the public perception of EA of the other community situations?
To add to folks disagreeing with the "size of numbers", from my perspective:
Most respondents to Rethink's survey hadn't encountered EA. Of those who had (233), only 18 (1.1% of total respondents) referred to FTX/SBF explicitly or obliquely when asked what they think effective altruism means or where and when they first heard of it.
I think that number is importantly 7.7% of all the people who had heard of EA, which seems not that small to me (though way smaller than my immersed-all-the-time-in-meta/FTX-stuff brain might have generated on its own when that was where my head was at).
And neither "what they think effective altruism means or where and when they first heard of it" is likely to capture all -- or perhaps even most -- respondents whose opinion of EA has been downgraded by the scandal, or who haven't heard about FTX yet but will downgrade once SBF's trial gives another big round of publicity.
More probative would be responses to something like: "Do you have any concerns or negative opinions about EA, and if so what are they?"
Thanks so much for doing this. I'm very happy to see how the general public and university students seemed to be mostly unaware and unaffected by FTX. By being happy, I don't mean to imply that we should take the situation lightly and not learn from it. I'm curious about other groups such as young professionals. However, I am somewhat shocked to see the massive drop in trust in leadership (1/3 distrusting leaders). This is definitely a significant effect which might yield good consequences - e.g., people being more likely to develop their own views and be less inclined to defer to certain individuals.
As part of the EA Strategy Fortnight, we want to compile some survey results addressing how FTX has impacted the EA brand—particularly people’s sentiments towards EA. From the research we have on attitudes toward EA from the EA community, the general public, and university students, it seems that the FTX crash hasn’t, overall, impacted sentiments toward EA very much.
This is not to say that FTX has not significantly impacted people in many ways, including mental and emotional health, levels of trust in EA leadership, and increased uncertainty in EA as a movement. We hope compiling these survey results will help contextualize our individual experiences and improve our understanding of general attitudes toward EA.
In December 2022, Rethink Priorities ran a survey to gauge how the FTX crisis affected EA community members’ attitudes toward the EA movement, organizations, and leadership (fuller results of that survey from Rethink are here).[1]
Data points:
The CEA groups team surveyed some university group organizers regarding the states of their individual groups in May 2023. From November 2022 to February 2023, Rethink commissioned a survey to measure campus awareness of EA and whether students’ awareness of EA was because of FTX.[2]
Data points:
Rethink ran surveys pre- and post-FTX assessing awareness of and feelings towards EA in the US general public and later in more elite (educated and informed) US groups in February-March (post-FTX).
Data points:
While not a formal survey, we thought it might be useful to include some info about how major EA donors have responded. This is primarily anecdotal information from people fundraising for EV-related entities.
This post is part of EA Strategy Fortnight. You can see other Strategy Fortnight posts here.
Rethink plans on conducting a followup survey to this survey, as referenced here.
Our understanding is that Rethink is planning to publish more information about this and the next survey soon
Great, this is useful data.
That's a big drop! In practice I've only ever seen this type of satisfaction scale give results between about 7/10 through 9.5/10 (which makes sense, right, if my satisfaction with EA is 3/10 then I'm probably not sticking around the community and answering member surveys), so that decline is a real big chunk of the scale's de facto range.
I suppose it's not surprising that the impact on perception is much bigger inside EA, where there's (appropriately) been tons of discourse on this, than in the general public.
We can get a better intimation of the magnitude of the effect here with some further calculations. If we take all the people who have pre and post FTX satisfaction responses (n = 951), we see that 4% of them have a satisfaction score that went up, 53% remained the same, and 43% went down. That’s quite a striking negative impact. For those people whose scores went down, 67% had a reduction of only 1 point, 22% of 2 points, and then 7%, 3%, and 1% each for -3, -4, and -5 points.
We can also try to translate this effect into some more commonly used effect size metrics. Firstly, we can utilise a nice summary effect size metric for these ratings known as probability of superiority (PSup), which makes relatively few assumptions about the data - mainly that higher ratings are higher and lower ratings are lower, within the same respondent. This metric summarises the difference over time by taking the proportion of cases in which a score was higher pre-FTX (42.7%), and assigning a 50% weight to cases in which the score was the same from pre to post FTX (.5 * 53.2% = 26.6%), and adding these quantities together (69.3%). This metric is taken as an approximation of the proportion of people who would report being more satisfied before vs. after in a forced choice of being more or less satisfied. If everyone was more satisfied before, PSup would be 100%, if everyone was more satisfied after, PSup would be 0, and if it were just as likely for people to be more or less satisfied before or after, PSup would be 50%. In this case, we get a PSup of 69.3%. This corresponds to an effect size in standard deviation units (like Cohen’s d), of approximately .7.
We would encourage people not to just look up whether these are small or large effects in a table that would say e.g, from wikipedia, that .7 is in the ‘medium’ effect size bin. Think about how you would respond on this kind of question, what a difference of 1 or more points would mean in your head, and what precisely you think the proportions of people giving different responses substantively might mean to them. How one can best interpret effect sizes varies greatly with context
Thanks!
I think this is a reasonable hypothesis, but there are also effects in the opposite direction (e.g. people who don't care that much about EA don't bother to track things very closely). Indeed, in this survey more engaged respondents were more concerned about EA leadership, not less.
Overall, Rethink didn't find any difference in change in satisfaction based on engagement level. I'm not sure how the mechanism you propose affects things on net, but I definitely agree that FTX has had a greater impact on perception within EA rather than outside of it.
Nice point! I guess one should also have in mind that the 0.5 to 1 point decrease will not be permanent. I guess the effect of FTX on satisfaction in 2 years will be negligible, but I do not know.
(Obviously we can't put things on 0-10 scales but) I just want to add that a 0.5/10 decrease should be considered a medium-big drop.