H

Habryka

18772 karmaJoined Sep 2014

Bio

Project lead of LessWrong 2.0, often helping the EA Forum with various issues with the forum. If something is broken on the site, it's a good chance it's my fault (Sorry!).

Comments
1171

This link's hypothesis is about people just trying to fit in―but SBF seemed not to try to fit in to his peer group! He engaged in a series of reckless and fraudulent behaviors that none of his peers seemed to want.

(Author of the post) My model is that Sam had some initial tendencies for reckless behavior and bullet-biting, and those were then greatly exacerbated via evaporative cooling dynamics at FTX. 

It sounds like SBF drove away everyone who couldn't stand his methods until only people who tolerated him were left. That's a pretty different way of making an organization go insane.

Relatedly, this kind of evaporative cooling is exactly the dynamic I was trying to point to in my post. Quotes: 

People who don’t want to live up to the demanding standard leave, which causes evaporative cooling and this raises the standards for the people who remain. Frequently this also causes the group to lose critical mass. 

[...]

My current best model of what happened at an individual psychological level was many people being attracted to FTX/Alameda because of the potential resources, then many rounds of evaporative cooling as anyone who was not extremely hardcore according to the group standard was kicked out, with there being a constant sense of insecurity for everyone involved that came from the frequent purges of people who seemed to not be on board with the group standard.

Do you have links to people being very worried about gray goo stuff?

(Also, the post you link to makes this clear, but this was a prediction from when Eliezer was a teenager, or just turned 20, which does not make for a particularly good comparison, IMO)

Also, I don't think "completely uninvestigated" is a correct characterization -- they were investigated enough to be presented to a grand jury, which indicted SBF for campaign-finance violations. Federal prosecutors do not generally indict without a pretty good investigation first, especially in high-profile cases. I think we have a pretty decent idea of what he did (see pp. 18-22 of the prosecution's sentencing memo). Moreover, Salame and Singh -- who don't have extradition-related issues -- pled guilty to campaign-finance violations.

Oh, that is interesting and an update for me. I had interpreted the relevant section of the memo as more of a "we didn't really investigate it, but here is what we have", but you know this stuff much better than I do.

I... think I will continue describing this as "weird" though it makes sense that as a lawyer it's not that weird to you.

It feels very off to have a bunch of crimes uninvestigated because someone fled the country and then was extradited, and I am pretty confused why the Bahamas cooperated with Sam here (my guess is that it's a case of political corruption, though I can also imagine other reasons). It's not like the Bahamas had anything obvious to gain from Sam not being convicted of the campaign finance violations.

This is great!

Note that this list is not comprehensive. In-particular it doesn't go into detail on any of the campaign finance violations and other policy-stuff that Sam was involved in, which I think never ended up investigated due to a kind of weird agreement with the Bahamas authorities. But during the trial we did hear some pretty clear evidence of at least campaign finance violations (and my guess is there would be a bunch more if one kept digging, as well as stuff that wouldn't necessarily be crimes but stuff I think most people here would still consider highly unethical).

It seems to me that a case study of how exactly FTX occurred, and where things failed, would be among one of the best things to use to figure out what thing to do instead. 

Currently the majority of people who have an interest in this are blocked by not really knowing what worked and didn't work in the FTX case, and so probably will have trouble arguing compellingly for any alternative, and also lack some of the most crucial data. My guess is you might have the relevant information from informal conversations, but most don't. 

I do think also just directly looking for an alternative seems good. I am not saying that doing an FTX investigation is literally the very best thing to do in the world, it just seems better than what I see EA leadership spending their time on instead. If you had the choice between "figure out a mechanism detecting and propagating information about future adversarial behavior" and "do an FTX investigation", I would feel pretty great about both, and honestly don't really know which one I would prefer. As far as I can tell neither of these things is seeing much effort invested into it.

My current sense is that there is no motivation to find an alternative because people mistakenly think it works fine enough and so there is no need to try to find something better (and also in the absence of an investigation and clear arguments about why the rumor thing doesn't work, people probably think they can't really be blamed if the strategy fails again)

Habryka
4d56
10
3
3
3

I think investing in FTX was genuinely a good idea, if you were a profit maximizer, even if you strongly suspected the fraud. As Jason says, as an investor losing money due to fraud isn't any worse than losing money because a company fails to otherwise be profitable, so even assigning 20%-30% probability to fraud for a high-risk investment like FTX where you are expecting >2x returns in a short number of years will not make a huge difference to your bottomline.

In many ways you should expect being the kind of person who is willing to commit fraud to be positively associated with returns, because doing illegal and fradulent things means that the people who run the organization take on massive risk where you are not exposed to the downside, but you are exposed to the upside. It's not worth it to literally invest in fraud, but it is worth it to invest into the kind of company where the CEO is willing to go to prison, since you don't really have any risk of going to prison, but you get the upside of the legal risk they take on (think of Uber blatantly violating laws until they established a new market, which probably exposed leadership to substantial legal risk, but investors just got to reap the profits).

I'm not the person quoted, but I agree with this part, and some of the reasons for why I expect the results of an investigation like this to be boring aren’t based on any private or confidential information, so perhaps worth sharing.

One key reason: I think rumor mills are not very effective fraud detection mechanisms.

Huh, the same reason you cite for why you are not interested in doing an investigation is one of the key reasons why I want an investigation. 

It seems to me that current EA leadership is basically planning to continue a "our primary defense against bad actors is the rumor mill" strategy. Having an analysis of how that strategy did not work, and in some sense can't work for things like this seems like it's one of the things that would have the most potential to give rise to something better here.

Becca, Nicole and Max all stand out as people who I think burned out trying to make things go better around FTX stuff. 

Also Claire leaving her position worsened my expectations of how much Open Phil will do things that seem bad. Alexander also seems substantially worse than Holden on this dimension. I think Holden was on the way out anyways, but my sense was Claire found the FTX-adjacent work very stressful and that played a role in her leaving (I don't thinks she agrees with me on many of these issues, but I nevertheless trusted her decision-making more than others in the space).

Load more