Comments
It’s in the image on the lower far right— “After Bankman-Fried, effective altruists won’t be fooled again, Opinion by Zach Robinson.”
More from the author
Curated and popular this week
This is the third in a sequence of posts taken from my recent report: Why Did Environmentalism Become Partisan?
Summary
Rising partisanship did not make environmentalism more popular or politically effective. Instead, it saw flat or falling overall public opinion, fewer major legislative achievements, and fluctuating executive actions.
Public Opinion...
I think right now EAs might be making a significant mistake by paying insufficient attention to the political realm. As EAs we tend to figure out what’s most impactful for us to work on and focus hard. That’s great! But there are various actions that are ‘non-delegatable’ - the extent to which an individual can do the action is limited (like voting, going to a protest, making hard money contributions to particular campaigns). It might be useful if we were all more in the habit of doing variou...
Bentham’s Bulldog recently argued that AI won’t definitely make factory farms obsolete. I agree, but I’d go further and argue that by default AI won’t make factory farms obsolete. However, I think it’s possible (though not guaranteed) that AI could make factory farms a lot more humane.
He throws out an 80% chance of cultivated meat being developed, and a 70% chance of it displacing factory far...
Recent opportunities to take action
The Washington Post article rings quite hollow. It claims that CEA and other EA organizations have taken FTX's downfall as an opportunity for "reflection and institutional reform", and cites the legal investigation CEA sponsored as evidence of this.
However, as far as I can tell the primary goal of that legal investigation was mostly PR-related,[1] trying to clear CEA's name and prove to the outside world that no one knew the full extend of FTX's fraud, and was not aimed at facilitating an internal reflection process (or at the very least people I've talked to at CEA described it to me as something they did not at all expect to be helpful as part of a reflection process, and multiple described the constraints imposed by it as harmful).
If-anything the legal investigation seems to have actually substantially interferred with CEA reflection processes, with Will MacAskill himself telling me that the EV board prevented him from publishing his reflections on FTX due to legal and PR concerns, and due to it maybe making the legal investigation seem less legitimate.
Overall, the piece reads like a puff piece and not like something that displays real reflection. I mean, it's fine to write puff-pieces from time to time, but it rings hollow in a world where actual institutional reform as a result of FTX has been almost completely absent.
Edit: I mean PR here not necessarily in a shallow sense. I think it's reasonable for CEA to want to clear its name. It just has basically nothing to do with an "internal reflection process" and "institutional reform" and claiming it does seems deceptive to me
I think you're missing some important ground in between "reflection process" and "PR exercise".
I can't speak for EV or other people then on the boards, but from my perspective the purpose of the legal investigation was primarily about helping to facilitate justified trust. Sam had by many been seen as a trusted EA leader, and had previously been on the board of CEA US. It seemed it wouldn't be unreasonable if people in EA (or even within EV) started worrying that leadership were covering things up. Having an external investigation was, although not a cheap signal to send, much cheaper in worlds where there was nothing to hide compared to worlds where we wanted to hide something; and internal trust is extremely important. Between that and wanting to be able to credibly signal to external stakeholders like the Charity Commission, I think general PR was a long way down the list of considerations.
Yep, I think the investigation did screen off some extreme scenarios in which people at CEA were colluding very actively with Sam about his biggest crimes. I at least had very little probability on this, and while it's good to clear up such an extreme scenario, I don't think it has much to do with actually preventing future things like FTX.
I totally agree there is value in signaling to external stakeholders that CEA is not literally criminal, but I don't think this has anything to do with a claimed "internal reflection process" and "institutional reform".
If the section had said "my organization has been working hard on trying to clarify that we were not literally colluding on crimes with Sam" then I wouldn't have objected, or even something weaker like "I do think people overestimate the degree to which EA knew about Sam's crimes, and a recent investigation showed that at CEA nobody knew of the major crimes that Sam committed before FTX collapsed". But that's not what the section claims. It directly claims that the investigation was part of an "internal reflection process" and "instutitional reform", and I have been shared on documents by CEA employees where the legal investigation was explicitly called out as not being helpful for facilitating a reflection process and institutional reform (which, to be clear, does not mean that others at CEA don't think it was important, but I think the case for that is quite weak, given the large costs it imposed on other people trying to do investigations, aggregating evidence, and the extreme narrowness of the questions investigated).
Sorry, I wasn't meaning to object to your critique overall. On my impression it's substantively correct that it's a puff piece and slightly disingenuous. (Maybe I should have owned that up-front. I still feel a bit bad about saying it baldly, because Zach is relatively new in a role that I think is often both difficult and thankless; I'd guess he's doing a good job overall and it seems bad if the feedback he gets skews too negative; I think it's probably good if he does more media. But I'm now worried that not saying it will be more confusing/costly, so I've just said it.)
But:
Yeah, I agree with this.
I think Zach definitely has a very difficult job, and I would like to support him in what he is doing. I seem to have disagreements about how to go about it, but that doesn't change the fact that I think he is paying large costs in stress and associated suffering in the pursuit of aims that I do care deeply about, and I think it's important to recognize that despite any disagreements we might have about strategy and tactics. Man in the arena and all that.
So, I feel like you are using "improving legibility" as kind of a euphemism here. I don't currently believe that you think that the changes that CEA has made would make it easier for you or me or the external world to discover whether something like FTX was likely to happen again. At least I can't think of any changes that would help with that.
To the contrary, my sense is the changes that have happened are much more directly optimizing for "making people believe that something like FTX won't happen again", but not in a way that is actually sensitive to the underlying facts. Like, I don't think CEA has made it easier for external parties to judge whether EA will have good or bad consequences for the world.
I think in order to use the words "improving legibility" here, the key thing that needs to be the case is that CEA is doing things that would cause people to believe that future things like FTX are likely to happen, if they are likely to happen, and to cause people to believe that future things like FTX are unlikely to happen, if they are indeed are unlikely to happen. I don't currently see such things, and indeed my guess is the public's ability to judge what is going in inside of EA, and the consequences of our actions, has gotten worse, not better, as a result of recent changes.
I don't think the legal investigation helps very much here, and I am not sure whether it overall would make people's beliefs here more or less accurate (as I also think the similar OpenAI investigation has demonstrated) and the primary reason people do these kinds of legal investigations is to get a voice of authority on their side that makes people more hesitant to criticize them. I don't think it's completely useless, but it's pretty close to it (especially when realizing how much OpenAI seems to have been able to manipulate a similar investigation in whatever way they desired).
(Sorry you're getting downvoted, this seems like a productive conversation to me.)
There's a big question about who things are being legible to. I don't think this would make things meaningfully more legible to you or people in your reference class of basically being high context on the org and having a lot of intersections in social circles.
I think there's a large crowd who are closer to "the general public" who only know about things via stuff that's written publicly, and will only ever read a small fraction of that. I think that these kinds of reform help to make legible to that crowd that the org isn't on some broad index-of-corruption where there is a culture of doing things for your mates, following the path of convenience (especially if that means getting more money), etc. That's not directly about "will there be another SBF?" (base rates are surely low), but pretty informative anyway for a bunch of questions about how excited people will be to interact with the org/community.
I think if EV had had a high corruption culture (even if they didn't literally know about any crimes), it would have been much more costly to have an external investigation (and such an investigation would likely have shaken out differently). This doesn't exclude the possibility that there would have been better/cheaper ways to signal this. And perhaps there are other effects I'm not tracking; I'd be interested if you wanted to elaborate on the view that it makes it harder for the public to tell what's going on inside EA.
This is an interesting argument, thanks for making it. Would you mind explaining in a bit more depth? I would have thought the structure of the investigation, especially including the fact that the results were not published, would limit the cost of the investigation. As far as I'm aware the only thing outside observers have to go on is this paragraph, which appears to be from this guy:
Is there some significant difference you see between the EVF investigation and the OpenAI investigation?
In the world with high corruption, I'd have expected more individuals to be scared of what an investigation would turn up.
If the investigation found major issues which weren't publicly reported on, it would increase the fragility of EV's position, since anyone who had seen the investigation could become a whistleblower with a relatively incontrovertible case.
These don't totally exclude "there's a big collusion not to share the issues", but "major internal collusion" without also "was colluding with SBF" is getting to be a narrower target, and I think it would still feel fragile (such that the colluders might rationally prefer to avoid an investigation).
(Not commenting on the comparison with the OpenAI investigation because I haven't understood it thoroughly enough to feel I have good takes.)
I have the opposite reaction.
Background
To recap, Zach wrote about four months ago:
The information released from the investigation in this statement is exceedingly narrow -- we didn't have actual knowledge of SBF's criminal fraud. That's an exceedingly low bar to meet! That EVF won't be joining SBF on his Club Fed vacation is an awfully slender reed to talk about "justified trust."
General Reaction to Scope of Disclosure (and Likelihood of Possible Inferences)
Is there anything else that has been "credibly signal[ed]" through the investigation process? (quoting Owen's earlier comment). Although I am aware of the reasons to maintain legal privileges, the choice to do so does little to persuade that this is a world in which EVF has "nothing to hide compared to [one in which it] wanted to hide something" (quoting same comment). Indeed, worlds in which the investigation sponsor chooses to hide behind privilege as to all but one (or a few) of the report's ultimate conclusions seem very likely to be worlds in which there the sponsor "wanted to hide something." And the choice to exclude certain important questions from the investigation's scope would update me toward the view that the investigation sponsor didn't want to find out the answers or strongly suspected they would be unflattering.
If Mintz had concluded that there was no negligence or recklessness in relation to EVF's interactions with SBF, I am guessing that conclusion would have been disclosed! Likewise, if I wanted to convey to the CC that I was a trustworthy public charity, I would release a whole lot more than "no actual knowledge of a massive fraud" if I could do so.
Reacting to the Positive View of the Investigation
But fear based on what an investigation would turn up only makes sense if you were concerned that the negative results would get out. So I think this is only correct insofar as those conducting the investigation were confident that it would find no actual knowledge of the criminal fraud.
But the number of people who would see the investigation report could be extremely limited. Presumably access to the report itself is extremely limited, given the rationale about not waiving privileges and protecting sources who spoke to the investigative firm. Outside counsel can't leak without counsel surrendering his law license, incurring massive liability, and watching his business crumble. It's plausible to me that no more than 10-15 people at EVF have seen the full report, and that these people are almost all insiders who are rather unlikely to leak.
How The Investigation Could Have Actually Rebuilt Lost Trust and Confidence
There was a more transparent / credible way to do this. EVF could have released, in advance, an appropriate range of specific questions upon which the external investigator was being asked to make findings of fact -- as well as a set of possible responses (on a scale of "investigation rules this out with very high confidence" to "investigation shows this is almost certain"). For example -- and these would probably have several subquestions each -- one could announce in advance that the following questions were in scope and that the investigator had committed to providing specific answers:
That kind of pre-commitment would have updated my faith in the process, and my confidence that the investigation reached all important topics. If EVF chose not to release the answers to those questions, it would have known that we could easily draw the appropriate inferences. Under those circumstances -- but not the actual circumstances -- I would view willingness to investigate as a valuable signal.
Thanks, this makes sense. I hadn't thought about the possibility of committing to have the external investigation provide public answers on specific questions. (And the fact that I hadn't thought of it gives me something to reflect on about how to ideally act during crises.)
Yeah, I think you are pointing towards something real here.
Like, I do think a thing that drove my reaction to this was a perspective in which it was obvious that most people in EA didn't literally actively participate in the FTX fraud. I have encountered very extreme and obviously wrong opinions about this in the public (the comment section of the WaPo article provides many examples of this), and there is some value in engaging with that.
But I do think that is engaging with a position that is extremely shallow, and the mechanism of it seems like its trying to attack shallowness with more shallowness, which I don't think is necessarily a strategy to rule out, but one that I am glad EA has avoided so far.
The way I would like Zack to have done it would have been to make a well-scoped statement that is honest and says "we might still be substantially responsible for SBF, but if so, we were because of indirect effects, not because we directly helped with the fraud. We really didn't know how bad the fraud was, and we did not directly participate in it, and here is an investigation that shows that"
I mean, as I mentioned as a concrete example, Will was told by CEA board members and leadership to not write anything detailed publicly about his FTX experiences due to the ongoing investigation. This seems like it has been extremely costly in terms of my ability and others ability to understand what happened, and I don't see the corresponding payoff in the investigation results. To quote from Will 9 months ago:
Will publishing backwards-looking posts on FTX seems like the obvious and most central thing that should happen as part of FTX reform. We have clear documentation that the investigation prevented that from happening (and as far as I can tell the reflections are still not published, so it seems like the investigations completely prevented this kind of information from entering the public record).
This seems like the headline claim to me. EAs should not be claiming false things in the Washington Post, of all things.
I didn't intend the above to be a straightforward accusation of lying. My sense is that different people at CEA disagree here about how useful this investigation was for reform purposes (which I think is evidence of the claim being false, but not strong evidence of deception).
If I had to make a bet, I would say majority opinion at CEA on the question of "did this investigation obstruct or assist internal reform efforts at CEA (as opposed to e.g. address legal liability and reputational risks independent from their best guess about what appropriate reform is)?" would be "obstruct", but I don't know, I haven't done a survey.
When I was talking to people about the investigation in the context of broader reform, the almost universal response was "oh, I don't think you should get your hopes up about this investigation" and "oh, I don't really think the investigation is going to help much with that". And that seemed reflected in some Google Docs I saw. But idk, maybe Zack thinks it was more part of it.
We really need to see the receipts on any claim that a significant degree of actual institutional reform as a result of FTX has occurred. I think that's part of why the op-ed comes off as a puff piece. Another reason is that we haven't seen the receipts to sufficiently establish that there was meaningful investigation of any question beyond whether anyone at EVF had actual knowledge of the fraud.
The kinds of reform-with-receipts I'd be looking for might include:
By analogy, administrative law judges in the US federal government enjoy very strong protections from adverse actions by their employing agencies. The costs of these protections (e.g., shielding ALJs who are just plain awful at their jobs) are seen as less important than promoting their independence from the agency that has an interest in their judicial decisions.
I agree with this overall sentiment, though want to mark that I don't think "donor risk" is the right way to model this.
The key thing to IMO investigate is why SBF, substantially driven by EA principles, built FTX on a substantial basis of fraud. Even if FTX/Sam had donated nothing to CEA, the harm seems just as big, and the key thing to investigate is what role EA played in FTX coming into existence, not what EA could have done to avoid being associated with SBF.
I doubt we disagree much, if at all.
Examining "what role EA played in FTX coming into existence" seems desirable, but doing a good job on this still seems rather challenging as a practical matter at least for now. FTX is unlikely to cooperate (and doing so would consume revenue that morally belongs to the victims). SBF denies having done anything wrong and will appeal. The co-conspirators are not going to talk because of their deal with USAO/SDNY, and even once that is done, I'd probably be worried about state and foreign criminal exposure in their shoes.[1] Although EVF settled with the FTX estate, many actors and orgs still have legal exposure. EVF could have counsel investigate EVF and keep the results privileged, but a license to practice law doesn't give you the ability to investigate whatever you'd like and keep the results secret.
One final issue is who should sponsor an investigation of "what role EA played in FTX coming into existence"; given the linkages between EVF, EVF insiders, and SBF, EVF would be somewhere in the vicinity of last place on my ideal preference list.
As for the reference to "donor risk" in the reform-with-receipts segment, I had examples of the policy response I'd like to see specifically from EVF in mind. But I do think donor-risk management is part of a defense-in-depth strategy. No one owns EA principles, and while taking steps to discourage people away from fraud-approving views, we're not going to find any way to absolutely preclude people from interpreting / applying those principles in a way that motivates them to do awful things.[2]
That means we shouldn't put all the marbles into one basket. If a would-be fraudster knows ahead of time that EA orgs will reject his money if it seems too suspect, and that in fact the donor due diligence process might lead to discovery of the fraud --> life in a prison cell, then they are less likely to start fraud-to-give in the first place. There being no way to be 100.00% sure donor money is clean, I gestured to "donor risk issues" rather than overselling with words like assure, ensure, guarantee, etc.[3]
It's usually not considered cool for a state to prosecute after the feds, but it's not against the rules. And it's likely that the co-conspirators violated criminal laws in a number of countries where victims lived. I would not assume that the US, which can be pretty aggressive on seeking extradition for financial crimes where the defendants never set foot on U.S. soil, would deny such a request.
In the same way, no one owns any set of ideas, and almost all ideas worth having can be twisted to justify bad actions. Of course, I would definitely agree that EA ideas are unusually vulnerable to being transmuted into naive consequentialism.
As someone who is significantly less than 100% a utilitarian, I also identify a moral obligation to take reasonable steps to address the possibility that an organization might be receiving funds tainted by fraud (among other things).
I think CEA or EV collaborating with an external but still within-EA trusted organization seems like the best choice to me here. Hiring someone who is broadly known to be independent (like, IDK, you could choose someone from Rethink, or Tyler Cowen, or someone else in that kind of reference class), seems like a good idea.
[Edit: With "collaboration" as defined in Habryka's response below, my question dissolves.]
Is there a reason to prefer CEA or EV collaborating with an investigator versus someone (or several someones) funding an investigator(s), taking a back seat once the proposal is accepted and funded, and deferring to the investigator(s) what to publish? [I may be reading too much into "collaborating" in your first sentence.]What I mean by collaboration is "is willing to share any information with them and allow staff to speak freely". The key obstacle I have faced in trying to do investigations is that nobody is willing to talk or say anything that goes on the record by organizational policy, which of course makes this kind of thing very hard to pull off.
I also think it would help a lot if CEA were to lend some credibility to the investigation. People don't want to repeat the same thing hundreds of times, and it would IMO be good for CEA/EV to put some social capital on the line to encourage people to talk to the investigators.
Considering how much mud was being slung around the FTX collapse, "clearing CEA's name" and proving that no one there knew about the fraud seems not just like PR to me, but pretty important for getting the org back to a place where it’s able to meaningfully do its work.
Plus, that investigation is not the only thing mentioned in the reflection reform paragraph. The very next sentence also says CEA has "reinvested in donor due diligence, updated our conflict-of-interest policies and reformed the governance of our organization, replacing leadership on the board and the staff."
I don't know of any staff that was let go as a result of FTX reflections (and I have asked about this repeatedly). Many people quit, but nobody was fired for any FTX things among leadership, and nobody who quit would have been fired. There is some small chance I am missing some supposed staff changes here, but claiming that CEA "replaced leadership on the staff" as a result of FTX seems straightforwardly false (though if there was something behind the scenes that I don't know about, I would love to hear it, but I currently disbelieve the bolded section).
The rest of the statements here seem pretty vacuous and almost impossible to falsify, and very hard distinguish from being done for PR reasons as opposed to genuine reflection. Briefly going through the areas where change is claimed:
Donor due diligence: I mean, I don't think the right lesson to take away from FTX is to be much more hesitant about accepting money from people. The key thing to understand is why EA seems to have created FTX in the first place. So I don't see the relevance of this. Yes, accepting the money was bad PR, I don't think it was bad for the world (there is some decision-theoretic argument here that maybe by refusing to accept money from bad people would have disincentivized the bad things happening, but I think that's very weak).
Conflict-of-interest policies: This seems maybe real. Conflicts of interest did possibly play a substantial role in FTX, but it really doesn't seem like the primary dynamic going on. I do struggle to understand how anything like CEA-internal conflict of interest policies would have helped with anything like FTX.
I also haven't seen these conflict of interest policies, and judging whether they seem like meaningful reform would require engaging with the details.
Reformed the governance of our organization: EV is shutting down, which seems like the biggest governance reform. It is the case that EV was a huge legal mess, and that FTX did seem like mild like evidence that you should have fewer shell companies and stuff, and maybe value standard operating practices more. It's not completely unrelated, but I really struggle to see how "things like EV don't exist" would somehow have made a difference for FTX happening.
Within CEA, I don't know of any governance reform that seem downstream of any FTX reflection, and I have talked to many CEA employees about this. If there are substantial governance changes downstream of learning from FTX, then I would love to hear them, but none such changes have been announced and when I have asked the literal question "why has nothing happened at CEA as a result of FTX?" people have not usually responded with "oh, what do you mean, we have made substantial governance changes like X".
Overall, I don't think Zack gets to claim any of the things in this paragraph as substantial evidence of reform or of a substantial internal reflection process. I think the COI policy maybe is real, but I don't know the details and I presently doubt it.
(Edit: I realized I forgot that Zack also refered to changes to the board composition. Those do seem more real, with both Nick and Will stepping down. My best sense is that Nick would have been a good board member and better than alternatives, so I would take that as a step backwards, not forwards, despite his FTX ties. Overall my sense has been that changes to the board have been primarily driven by burnout and stress related to FTX, and not the result of FTX-adjacent reform. At least I haven't heard of any plans or any intentional selection of board members with specific FTX-reform adjacent ideas in mind. But overall I do think that "changes in board composition were part of an internal reform effort" is the most defensible part of that paragraph and doesn't seem disingenuous the way the others seem to me)
'I mean, I don't think the right less to take away from FTX is to be much more hesitant about accepting money from people. The key thing to understand is why EA seems to have created FTX in the first place.'
In your view, what are the right lessons and why did EA create FTX?
I've written a post about some of my best guesses here:
https://forum.effectivealtruism.org/posts/MMM24repKAzYxZqjn/my-tentative-best-guess-on-how-eas-and-rationalists
I also have lots of other thoughts and would love to talk more about the details with people, though this specific comment thread seems probably not the right place for it.
I don't love this article but it's fine. In general many other articles about EA are too negative so it doesn't really seem worth writing a big correction when the median person who hears about EA probably hears about the right thing.
Specifically, are new readers gonna believe that EA has done a load of useful soul-searching because this articles says it? I doubt it. There are enough articles saying that EAs are a bunch of cynical psychopaths that many will probably assume this is the fluff piece (that it is).
I don't really think this meta discussion is that high priority, the better question is, how does EA improve and focus back on effective altruism, but I don't have a great answer to that, other than write articles specifically focused at that.