2237 karmaJoined Sep 2014


I don't think we would have been able to use the additional information we would have gained from delaying the industrial revolution but I think if we could have the answer might be "yes". It's easy to see in hindsight that it went well overall, but that doesn't mean that the correct ex ante attitude shouldn't have been caution!

100% agree. I think it is almost always better to be honest, even if that makes you look weird. If you are worried about optics, "oh yeah, we say this to get people in but we don't really believe it" looks pretty bad.

I would qualify this statement by saying that it would be nice for OP to have more reasoning transparency, but it is not the most important thing and can be expensive to produce. So it would be quite reasonable for additional marginal transparency to not be the most valuable use of their staff time.

  • Some EAs knew about his relationship with Caroline, which would undermine the public story about FTX<->Alameda relations, but didn't disclose this.
  • Some EAs knew that Sam and FTX weren't behaving frugally, which would undermine his public image, but also didn't disclose.

FWIW, these examples feel hindsight-bias-y to me. They have the flavour of "we now know this information was significant, so of course at the time people should have known this and done something about it". If I put myself in the shoes of the "some EAs" in these examples, it's not clear to me that I would have acted differently and it's not clear what norm would suggest different action.

Suppose you are a random EA. Maybe you run an EA org. You have met Sam a few times, he seems fine. You hear that he is dating Caroline. You go "oh, that's disappointing, probably bad for the organization, but I guess we'll have to see what happens" and get on with your life.

It seems to me that you're suggesting this was negligent, but I'm not sure what norm we would like to enforce here. Always publish (on the forum?) negative information about people you are at all associated with, even if it seems like it might not matter? 

The case doesn't seem much stronger to me even if you're, say, on the FTX Foundation board. You hear something that sounds potentially bad, maybe you investigate a little, but it seems that you want a norm that there should be some kind of big public disclosure, and I'm not sure that really is something we could endorse in general.

To reuse your example, if you were the only person the perpetrator of the heist could con into lending their car to act as a getaway vehicle, then that would make P(Heist happens | Your actions) quite a bit higher than P(Heist happens | You acting differently), but you would still be primarily a mark or (minor) victim of the crime

Yes, this is a good point. I notice that I don't in fact feel very moved by arguments that P(FTX exists | EA exists) is higher, I think for this reason. So perhaps I shouldn't have brought that argument up, since I don't think it's the crux (although I do think it's true, it's just over-determining the conclusion).

Only ~10k/10B people are in EA, while they represent ~1/10 of history's worst frauds, giving a risk ratio of about 10^5:1, or 10^7:1, if you focus on an early cohort of EAs.

This seems wildly off to me - I think the strength of the conclusion here should make you doubt the reasoning!

I think that the scale of the fraud seems like a random variable uncorrelated with our behaviour as a community. It seems to me like the relevant outcome is "producing someone able and willing to run a company-level fraud"; given that, whether or not it's a big one or a small one seems like it just adds (an enormous amount of) noise. 

How many people-able-and-willing-to-run-a-company-level-fraud does the world produce? I'm not sure, but I would say it has to be at least a dozen per year in finance alone, and more in crypto. So far EA has got 1. Is that above the base rate? Hard to say, especially if you control for the community's demographics (socioeconomic class, education, etc.).

I do think it's an interesting question whether EA is prone to generate Sams at higher than the base rate. I think it's pretty hard to tell from a single case, though.

I am one of the people who thinks that we have reacted too much to the FTX situation. I think as a community we sometimes suffer from a surfeit of agency and we should consider the degree to which we are also victims of SBF's fraud. We got used. It's like someone asking to borrow your car and then using it as the getaway car for a heist. Sure, you causally contributed, but your main sin was poor character judgement. And many, many people, even very sophisticated people, get taken in by charismatic financial con men.

I also think there's too much uncritical acceptance of what SBF said about his thoughts and motivations. That makes it look like FTX wouldn't have existed and the fraud wouldn't have happened without EA ideas... but that's what Sam wants you to think.

I think if SBF had never encountered EA, he would still have been working in finance and spotted the opportunity to cash in with a crypto exchange. And he would still have been put in situations where he could commit fraud to keep his enterprise alive. And he would still have done it. Perhaps RB provided cover for what he was thinking (subconsciously even), but plenty of financial fraud happens with much flimsier cover.

That is, I think P(FTX exists) is not much lower than P(FTX exists | SBF in EA), and similarly for P(FTX is successful) and P(SBF commits major fraud) (actually I'd find it plausible to think that the EA involvement might even have lowered the chances of fraud).

Overall this makes me mostly feel that we should be embarrassed about the situation and not really blameworthy. I think the following people could do with some reflection:

  • People who met, liked, and promoted SBF. "How could I have spotted that he was a con man?"
  • People who bought SBF's RB reasoning at face value. "How could I have noticed that this was a cover for bad behaviour?"
    • I include myself in this a bit. I remember reading the Tyler Cowen interview where he says he'd play double or nothing with the world on a coin flip and thinking "haha, how cute, he enjoys biting some bullets in decision theory, I'm sure this mostly just applies to his investing strategy". 

That's about it. I think the people who took money from the FTXFF are pretty blameless and it's weird to expect them to have done massive due diligence that others didn't manage.

It seems like we could use the new reactions for some of this. At the moment they're all positive but there could be some negative ones. And we'd want to be able to put the reactions on top level posts (which seems good anyway).

I wonder if the focus on "narrow EA" is a reflection of short AI timelines and/or a belief that we need to make changes sooner rather than later.

It seems to me that "global EA" looks better the longer the future we have. Gains in people compound, and other countries may be much more influential in the future. If Nigeria is a global power in 50 years, then growing a community there now might be a good investment.

I don't think this has a clear answer though, since benefits from actually solving problems sooner can compound too.

Load more