The funding conversation we left unfinished

Thanks for restarting this conversation!

Relatedly, it's also time to start focusing on the increased conflicts of interest and epistemic challenges that an influx of AI industry insider cash could bring. As Nathan implies in his comment, proximity to massive amounts of money can have significant adverse effects in addition to positive ones. And I worry that if and when a relevant IPO or cashout is announced, the aroma of expected funds will not improve our ability to navigate these challenges well.

Most people are very hesitant to bite the hand that feeds them. Orgs may be hesitant to do things that could adversely effect their ability to access future donations from current or expected donors. We might expect that AI-insider donors will disproportionately choose to fund charities that align fairly well with -- or at least are consonant with -- their personal interests and viewpoints.

(I am aware that significant conflicts of interest with the AI industry have existed in the past and continue to exist. But there's not much I can do about that, and the conflict for the hypothesized new funding sources seems potentially even more acute. I imagine that some of these donors will retain significant financial interests in frontier AI labs even if they cash out part of their equity, as opposed to old-school donors who have a lesser portion of their wealth in AI. Also, Dustin and Cari donated their Anthropic stake, which addresses their personal conflict of interest on that front (although it may create a conflict for wherever that donation went)). 

For purposes of these rest of this comment, a significantly AI-involved source has a continuing role at an frontier AI lab, or has a significant portion of their wealth still tied up in AI-related equity. The term does not include those who have exited their AI-related positions.

What Sorts of Adverse Effects Could Happen?

There are various ways in which the new donors' personal financial interests could bias the community's actions and beliefs. I use the word bias here because those personal interests should not have an effect on what the community believes and says.

Take stop/pause advocacy for an obvious example. Without expressing a view about the merits of such advocacy, significantly AI-involved sources have an obvious conflict of interest that creates a bias against that sort of work. To be fair, it is their choice on how to spend their money.

But -- one could imagine the community changing its behavior and/or beliefs in ways that are problematic. Maybe people don't write posts and comments in support of stop/pause advocacy because they don't want to irritate the new funders. Maybe grantmakers don't recommend stop/pause advocacy grants for their other clients because their AI-involved clients could view their money as indirectly supporting such advocacy via funging. 

There's also a risk of losing public credibility -- it would not be hard to cast orgs that took AI-involved source funds as something like a lobbying arm of Anthropic equity holders.

What Types of Things Could Be Done to Mitigate This?

This is tougher, but some low-hanging fruit might include:

• Orgs could commit to identifying whether, and how much, of their funding comes from significantly AI-involved sources.
• Many orgs could have a limit on the percentage of their budget they will accept from significantly AI-involved sources. Some orgs -- those with particular sensitivity on AI knowledge and policy -- should probably avoid any major gifts from AI-involved sources at all.
• Particularly sensitive orgs could be granted extended runways and/or funding agreements with some sort of independent protection against non-renewal.
• Other donors could provide more funding for red-teaming AI work, especially work that potentially affected AI-involved source donors.

Anyway, it is this sort of thing that concerns me more than (e.g.) some university student scamming a free trip to some location by simulating interest in EA.

A basic issue with a lot of deliberate philanthropy is the tension between:

• In many domains, much of the biggest gains are likely to come from marginal opportunities. E.g. because they have more value of information, more large upsides, more addressing neglected areas (and therefore plausibly strategically important.
• Marginal opportunities are harder to evaluate.
  • There's less preexisting understanding, on the part of fund allocators.
  • The people applying would tend to be less tested.
  • Therefore, it's easier to game.

The kneejerk solution I'd propose is "proof of novel work". If you want funding to do X, you should show that you've done something to address X that others haven't done. That could be a detailed insightful write-up (which indicates serious thinking / fact-finding); that could be some you did on the side, which isn't necessarily conceptually novel but is useful work on X that others were not doing; etc.

I assume that this is an obvious / not new idea, so I'm curious where it doesn't work. Also curious what else has been tried. (E.g. many organizations do "don't apply, we only give to {our friends, people we find through our own searches, people who are already getting funding, ...}".)

Sure, seems plausible. 

I guess I kind of like @William_MacAskill's piece or as much as I remember of it. 

My recollection is roughly this: 

• Yes, it's strange to have lots more money.
• Perhaps we're spending it badly.
• But also seeking not to spend enough money might be a bad thing, too.
• Frugal EA had something to recommend it.
• But more impact probably requires more resources. 

This seems good, though I guess it feels like a missing piece is: 

• Are we sure this money is got ethically?
• How much harm will getting this money for bad reasons hurt us? 

Also, looking back @trammell's takes have aged very well:

• It is unlikely we are in the most important time in history
• If not, it is good to save money for that time

Had Phil been listened to, then perhaps much of the FTX money would have been put aside, and things could have gone quite differently. 

So my non-EA friends point out that EAs have incentives to suck up to any group that are about to become rich. This seems something which I haven't seen a solid path through:

• It is much more effective to deal with the people who have the most money.
• It is hard to retain one's virtue while doing so. 

Having known, and had conflict with a number of wealthy people, it is hard to retain ones sense of integrity in the face of lifechanging funds. I've talked to SBF and even after the crash I felt a gravity that I didn't want to insult him lest he one day return to the heights of his influence. Sometimes that made me too cautious, sometimes, avoiding caution I was reckless. 

I guess in some sense the problem is that finding ways through uncomfortable situations requires sitting in discomfort, and I don't find EA to have a lot of internal battery for that kind of thing. Have we really resolved most of the various crises in a way that created harmony between those who disagreed? I'm not sure we have. So it's hard to be optimistic here. 

I find so much EA analysis, in general, to be too clever by half. (Per Wiktionary: "Shrewd but flawed by overthinking or excessive complexity, with a resulting tendency to be unreliable or unsuccessful.") So many conversations like this could be helped along by just having a simpler and more commonsense analysis. Does EA need to have a big conversation right now about how to handle it if EA suddenly gets tons of money? Probably not.

Expecting the money to come in sounds like wishful thinking. Even if there are Anthropic billionaires with liquidity in 2026 or 2027 (which is not guaranteed to happen), even if these billionaires are influenced by EA and want to give money to some of the same charities or cause areas as people in EA care about, who says the money is going to flow through the EA community/movement? If I were an Anthropic billionaire, rather than trying to be Sam Bankman-Fried 2.0 and just spraying a firehose of money at the EA community generally, I would pick the charities I want to donate to and give to them directly.

Besides Sam Bankman-Fried, the other billionaires who have donated to EA-related charities and causes like Dustin Moskovitz/Cari Tuna and Jaan Tallinn have completely managed their own giving. Sam Bankman-Fried’s behaviour in general was impulsive and chaotic — it seems like his financial crimes were less likely to be rational calculation and seem more like poor impulse control or a general disinhibition, as crime often is — and the way he gave money to EA seems like an extension of that. A more careful person probably wouldn’t do it that way. They would probably start a private foundation, hire some people to help manage it, and run it quietly out of the public view. Maybe they would take the unusual step of starting something like Open Philanthropy/Coefficient Giving and do their giving in a more public-facing way. But even so, this is still under their control and not the EA community’s control.

If some Anthropic billionaire does just back a truck full of money up to the EA community, that’s a good problem to have, and that’s the sort of problem you can digest and adapt to as it starts happening. You don’t need to invest a lot of your limited resources of time, energy, and attention to it 6 months to 3 years in advance, when it’s not actually clear it will ever happen at all. (This isn’t an asteroid, you don’t need to fret about long-tail risk.)