Part of the reason we are in this position is because nobody outside the EA community cares about AI safety enough to fund it or work in it. I’m very sympathetic to this; some of these close connections are inevitable.
However, it can also be true that these connections are completely unacceptable (edit), and that Anthropic should be trying harder than they are to find or create auditors that are genuinely independent. How do we do that?
I also agree in not letting the perfect be the enemy of the good. So surely having less constraints on the slowdown would make it more likely to succeed?
Could someone smart please explain how preserving a US-China capabilities gap is compatible with de-escalation or slowdowns? Wasn’t that like, the whole problem with the Cold War?
Then a line like ‘we did not intend to announce this, but decided to because it would have made it to light anyway’ would have sufficed, since there is a stated concern for transparency :)
Would you be able to speak more on why you decided to announce this publicly at all, rather than continue to use a pseudonym for Caroline and keep her away from outward-facing roles?
Why do you believe that the EA-wide and Manifund-specific harms you’ve already acknowledged, are worthwhile to incur, and to what end?
Do you believe there is some positive signalling value to be gained from announcing this that offsets the negative signalling?
Did you specifically trade off broader harms to EA’s reputation in order to advance Manifund’s own reputation? (My sense is Manifund has a non-overlapping target audience with EA)
I’m struggling to understand why this had to be announced, given the harms that specifically come from publicising it.
Sorry, could you clarify what the mechanism is for vulture deaths causing humans to die is? That’s really counterintuitive. Do you just mean that the chemicals killing vultures also kill humans?
Out of total curiosity: Many AIM seed grants can’t pay for two founders to take 90k a year, or at least, it would be unwise to. How are you thinking about the concern that if AIM pays sensibly for internal staff, that this could draw talent away from founding, which often can’t afford to pay as well? (I worry these aren’t separate talent pools, given the crossover between AIM staff and AIM founders!)
I don’t think the difference between $100m and $50B is that substantial for my point. Even though they are two orders of magnitude apart, I think the limit on how much money you should trust with someone who has no prior experience managing money is lower than that.
His creditors supplied the money—it is their judgement I’m questioning.
The losses, as you pointed out, happened due to highly leveraged bets. He didn't expect memory stocks to bleed as much as they did.
This is disqualifying if you are running a hedge fund. It’s literally in the name—you are supposed to hedge your positions in order to prevent an unexpected situation from tanking the whole fund.
Besides, it is possible that the fund will survive because his core thesis has paid off exceptionally well so far
We don’t judge funds by whether they don’t go bankrupt, we judge them by their performance against a market index over a long period of time. Even if the positions are net up by 2× or so, this is not particularly impressive in and of itself over a short period, because of survivorship bias. If you make a bunch of stupidly leveraged bets on different sectors, one of them is likely to pay off very well, but not for long, and not through downturns.
The AI sector has monotonically gone up since the release of ChatGPT—any overleveraged investor in this space would be likely to produce incredible gains. If one’s fund gets obliterated at the first market downturn because one was overleveraged, all this proves is that you managed your fund badly, not that you’re some kind of savant genius market whisperer.
(c.f. anything written about Cathie Wood in 2022—a lot of it has aged very poorly)
From SE Geyges: Is METR a meaningful check on Anthropic?.
Part of the reason we are in this position is because nobody outside the EA community cares about AI safety enough to fund it or work in it. I’m very sympathetic to this; some of these close connections are inevitable.
However, it can also be true that these connections are
completelyunacceptable (edit), and that Anthropic should be trying harder than they are to find or create auditors that are genuinely independent. How do we do that?I also agree in not letting the perfect be the enemy of the good. So surely having less constraints on the slowdown would make it more likely to succeed?
Right, then why risk the rhetoric of American superiority getting in the way of them signing a deal?
Could someone smart please explain how preserving a US-China capabilities gap is compatible with de-escalation or slowdowns? Wasn’t that like, the whole problem with the Cold War?
Then a line like ‘we did not intend to announce this, but decided to because it would have made it to light anyway’ would have sufficed, since there is a stated concern for transparency :)
I’m struggling to understand why this had to be announced, given the harms that specifically come from publicising it.
Sorry, could you clarify what the mechanism is for vulture deaths causing humans to die is? That’s really counterintuitive. Do you just mean that the chemicals killing vultures also kill humans?
Out of total curiosity: Many AIM seed grants can’t pay for two founders to take 90k a year, or at least, it would be unwise to. How are you thinking about the concern that if AIM pays sensibly for internal staff, that this could draw talent away from founding, which often can’t afford to pay as well? (I worry these aren’t separate talent pools, given the crossover between AIM staff and AIM founders!)
I don’t think the difference between $100m and $50B is that substantial for my point. Even though they are two orders of magnitude apart, I think the limit on how much money you should trust with someone who has no prior experience managing money is lower than that.
His creditors supplied the money—it is their judgement I’m questioning.
This is disqualifying if you are running a hedge fund. It’s literally in the name—you are supposed to hedge your positions in order to prevent an unexpected situation from tanking the whole fund.
We don’t judge funds by whether they don’t go bankrupt, we judge them by their performance against a market index over a long period of time. Even if the positions are net up by 2× or so, this is not particularly impressive in and of itself over a short period, because of survivorship bias. If you make a bunch of stupidly leveraged bets on different sectors, one of them is likely to pay off very well, but not for long, and not through downturns.
The AI sector has monotonically gone up since the release of ChatGPT—any overleveraged investor in this space would be likely to produce incredible gains. If one’s fund gets obliterated at the first market downturn because one was overleveraged, all this proves is that you managed your fund badly, not that you’re some kind of savant genius market whisperer.
(c.f. anything written about Cathie Wood in 2022—a lot of it has aged very poorly)