My takes on the FTX situation will (mostly) be cold, not hot

Here’s a followup with some reflections.

Note that I discuss some takeaways and potential lessons learned in this interview.

Here are some (somewhat redundant with the interview) things I feel like I’ve updated on in light of the FTX collapse and aftermath:

• The most obvious thing that’s changed is a tighter funding situation, which I addressed here.
• I’m generally more concerned about the dynamics I wrote about in EA is about maximization, and maximization is perilous. If I wrote that piece today, most of it would be the same, but the “Avoiding the pitfalls” section would be quite different (less reassuring/reassured). I’m not really sure what to do about these dynamics, i.e., how to reduce the risk that EA will encourage and attract perilous maximization, but a couple of possibilities:
  • It looks to me like the community needs to beef up and improve investments in activities like “identifying and warning about bad actors in the community,” and I regret not taking a stronger hand in doing so to date. (Recent sexual harassment developments reinforce this point.).
  • I’ve long wanted to try to write up a detailed intellectual case against what one might call “hard-core utilitarianism.” I think arguing about this sort of thing on the merits is probably the most promising way to reduce associated risks; EA isn’t (and I don’t want it to be) the kind of community where you can change what people operationally value just by saying you want it to change, and I think the intellectual case has to be made. I think there is a good substantive case for pluralism and moderation that could be better-explained and easier to find, and I’m thinking about how to make that happen (though I can’t promise to do so soon).
• I had some concerns about SBF and FTX, but I largely thought of the situation as not being my responsibility, as Open Philanthropy had no formal relationship to either. In hindsight, I wish I’d reasoned more like this: “This person is becoming very associated with effective altruism, so whether or not that’s due to anything I’ve done, it’s important to figure out whether that’s a bad thing and whether proactive distancing is needed.”
• I’m not surprised there are some bad actors in the EA community (I think bad actors exist in any community), but I’ve increased my picture of how much harm a small set of them can do, and hence I think it could be good for Open Philanthropy to become more conservative about funding and associating with people who might end up being bad actors (while recognizing that it won’t be able to predict perfectly on this front).
• Prior to the FTX collapse, I had been gradually updating toward feeling like Open Philanthropy should be less cautious with funding and other actions; quicker to trust our own intuitions and people who intuitively seemed to share our values; and generally less cautious. Some of this update was based on thinking that some folks associated with FTX were being successful with more self-trusting, less-cautious attitudes; some of it was based on seeing few immediate negative consequences of things like the Future Fund regranting program; some of it was probably a less rational response to peer pressure. I now feel the case for caution and deliberation in most actions is quite strong - partly because the substantive situation has changed (effective altruism is now enough in the spotlight, and controversial enough, that the costs of further problems seem higher than they did before).
  • On this front, I’ve updated a bit toward my previous self, and more so toward Alexander’s style, in terms of wanting to weigh both explicit risks and vague misgivings significantly before taking notable actions. That said, I think balance is needed and this is only a fairly moderate update, partly because I didn’t update enormously in the other direction before. I think I’m still overall more in favor of moving quickly than I was ~5 years ago, for a number of reasons. In any case I don’t expect there to be a dramatic visible change on this front in terms of Open Philanthropy’s grantmaking, though it might be investing more effort in improving functions like community health.
• Having seen the EA brand under the spotlight, I now think it isn’t a great brand for wide public outreach. It throws together a lot of very different things (global health giving, global catastrophic risk reduction, longtermism) in a way that makes sense to me but seems highly confusing to many, and puts them all under a wrapper that seems self-righteous and, for lack of a better term, punchable? I still think of myself as an effective altruist and think we should continue to have an EA brand for attracting the sort of people (like myself) who want to put a lot of dedicated, intensive time into thinking about what issues they can work on to do the most good; but I’m not sure this is the brand that will or should attract most of the people who can be helpful on key causes. I think it’s probably good to focus more on building communities and professional networks around specific causes (e.g., AI risk, biorisk, animal welfare, global health) relative to building them around “EA.”
• I think we should see “EA community building” as less valuable than before, if only because one of the biggest seeming success stories now seems to be a harm story. I think this concern applies to community building for specific issues as well. It’s hard to make a clean quantitative statement about how this will change Open Philanthropy's actions, but it’s a factor in how we recently ranked grants. I think it'll be important to do quite a bit more thinking about this (and in particular, to gather more data along these lines) in the longer run.

Would be interested in your (eventual) take on the following parallels between FTX and OpenAI:

1. Inspired/funded by EA
2. Taking big risks with other people's lives/money
3. Attempt at regulatory capture
4. Large employee exodus due to safety/ethics/governance concerns
5. Lack of public details of concerns due in part to non-disparagement agreements

Honestly, I’m happy with this compromise. I want to hear more about what ‘leadership’ is thinking, but I also understand the constraints you all have.

This obviously doesn’t answer the questions people have, but at least communicating this instead of radio silence is very much appreciated. For me at least, it feels like it helps reduce feelings of disconnectedness and makes the situation a little less frustrating.

Thank you for writing this - strong +1. At 80k we are going to be thinking carefully about what this means for our career advice and our ways of communicating - how this should change things and what we should do going forward. But there’s a decent amount we still don’t know and it will also just take time to figure that all out.

It feels like we've just gotten a load of new information, and there’s probably more coming, and I am in favour of updating on things carefully.

Holden - thanks very much for writing this; I strongly agree with the importance of patience, fact-gathering, wisdom, and cold takes. 

During a PR crisis, often the best communication strategy is not to communicate, and to let the media attention die down and move on to the next monetizable outrage narrative about somebody else or some other group that's allegedly done something wrong.

I would add just three supporting points.

First, hot takes tend to provoke hot counter-takes, leading to cycles of accusations and counter-accusations.  When a movement undergoes a moral crisis, and seems guilt-stricken, self-critical, and full of self-doubt, old grievances suddenly get aired, in hopes that the movement's members will be more vulnerable to various forms of moral blackmail, and will change their policies, norms, and ethos under conditions of high emotionality and time pressure. The hot takes and hot counter-takes can also escalate into clannish fractures and ideological schisms in the movement. In other words, any individual hot take might seem innocuous, but collectively, a barrage of hot takes flying in all directions can have huge negative side-effects on a movement's social cohesiveness and moral integrity, and can lead to changes that seem urgent and righteous in the short-term, but that have big hidden costs in the long term.

Second, any hot takes that are shared on EA Forum are in the public domain, and can be quoted by any journalist, pundit, muckraker, blogger, YouTuber, or grievance-holder, for any reason, to push any narrative they want. We are used to EA Forum seeming like a cozy, friendly, in-group medium for open and honest discussions. But in the present circumstances, we may need to treat EA Forum as a de facto EA public relations outlet in its own right. Everything we say on here can be taken, quoted out of context, misrepresented, and spun, by anybody out there who's hostile to EA. Thus, when writing our hot takes here, we might naively imagine the audience being the average EA reader -- rational, kind, constructive, sympathetic. But there's the tail risk that any given hot take will be weaponized by non-EAs to hurt EA in any way they can.

Third, some EA people seem to misunderstand the nature of PR issues, media narratives, and the 'brand equity' of social/activist/moral movements like EA. Certainly, as Holden notes, 'people who care too much about PR can be slippery and deceptive'. Many people outside the professions of PR, crisis management, market researcher, advertising, political campaigning, etc tend to view 'public relations' as very nebulous, vague, and unreal -- the realm of sociopathic mind-control wizards. 

However, public sentiment can be measured, quantified, analyzed, and influenced. Literally tens of thousands of people in market research do this all day, every day, for corporations, governments, activist movements, etc. There are facts of the matter about public perception of EA as a moral/social brand. Some actual number of people have heard about EA for the first time in the last few days -- maybe tens of millions. Some specific  % of them will have formed a negative, neutral, or positive impression of EA. Any negative impressions of EA will last an average of X days, weeks, or years. They will be Y% less (or more) likely to get involved in EA, or to donate money to EA. We don't know what those numbers actually are (though we should probably spend a bit of money on market research to find out how bad the damage has actually been.) 

There's a psychological reality to public sentiment -- however tricky it can be to measure, and however transient its effects can be. Most of us are amateurs  when it comes to thinking about PR. But it's better to recognize that we're newbies with a lot that we need to learn -- rather than dismissing PR concerns as beneath contempt.

I'm commenting here to say that while I don't plan to participate in public discussion of the FTX situation imminently (for similar reasons to the ones Holden gives above, though I don't totally agree with some of Holden's explanations here, and personally put more weight on some considerations here than others), I am planning to do so within the next several months. I'm sorry for how frustrating that is, though I endorse my choice.

Thanks for sharing your thoughts on this, especially your points on PR updated me a bit towards taking PR more seriously.

One piece of pushback on your overall message is that I think there are different kinds of communications than cold or hot takes (which I understand as more or less refined assessments of the situation and its implications). One can:

• share what one is currently doing about the situation,
• share details that help others figure out what can be learned from this,
  • (this sometimes might require some bravery and potentially making oneself susceptible to legal risks, but I'd guess that for many who can share useful info it wouldn't?)
• share your feelings about the crisis.

I'm overall feeling like contributing to the broader truth-seeking process is a generally cooperative and laudable move,  and that it's often relatively easy to share relatively robust assessments that one is unlikely to have to backtrack, such as those I've seen from MacAskill, Wiblin, Sam Harris.

For example I really appreciated Oliver Habryka reflecting publicly about his potential role in this situation by not sufficiently communicating his impression of SBF. I expect Habryka giving this "take" and the associated background info will not prove wrong-headed in 3 months, and it didn't seem driven by hot emotions or an overreaction to me.

Can you clarify which "public hearings" were demanded? Not sure if you're talking about how quickly the bankruptcy process has been moving at FTX, or how the reactions from people on EA Forum since the news about FTX started.

I'm waiting to read your take, especially since the conflict of interest issue has come up with FTX and you seem to have some conflicts of interest especially when it comes to AI safety funding . I'm curious how you have managed to stay unbiased.

I strongly agree with this post. Holden is choosing correctly here, in a difficult time. 

In my personal opinion, he has many intelligent takes and is quite handsome. I look forward to Holden's writing on AI risk.