This is a linkpost for https://www.goodventures.org/blog/an-update-from-good-ventures/
I wanted to share this update from Good Ventures (Cari and Dustin’s philanthropy), which seems relevant to the EA community.
Tl;dr: “while we generally plan to continue increasing our grantmaking in our existing focus areas via our partner Open Philanthropy, we have decided to exit a handful of sub-causes (amounting to less than 5% of our annual grantmaking), and we are no longer planning to expand into new causes in the near term by default.”
A few follow-ups on this from an Open Phil perspective:
- I want to apologize to directly affected grantees (who've already been notified) for the negative surprise here, and for our part in not better anticipating it.
- While this represents a real update, we remain deeply aligned with Good Ventures (they’re expecting to continue to increase giving via OP over time), and grateful for how many of the diverse funding opportunities we’ve recommended that they’ve been willing to tackle.
- An example of a new potential focus area that OP staff had been interested in exploring that Good Ventures is not planning to fund is research on the potential moral patienthood of digital minds. If any readers are interested in funding opportunities in that space, please reach out.
- Good Ventures has told us they don’t plan to exit any overall focus areas in the near term. But this update is an important reminder that such a high degree of reliance on one funder (especially on the GCR side) represents a structural risk. I think it’s important to diversify funding in many of the fields Good Ventures currently funds, and that doing so could make the funding base more stable both directly (by diversifying funding sources) and indirectly (by lowering the time and energy costs to Good Ventures from being such a disproportionately large funder).
- Another implication of these changes is that going forward, OP will have a higher bar for recommending grants that could draw on limited Good Ventures bandwidth, and so our program staff will face more constraints in terms of what they’re able to fund. We always knew we weren’t funding every worthy thing out there, but that will be even more true going forward. Accordingly, we expect marginal opportunities for other funders to look stronger going forward.
- Historically, OP has been focused on finding enough outstanding giving opportunities to hit Good Ventures’ spending targets, with a long-term vision that once we had hit those targets, we’d expand our work to support other donors seeking to maximize their impact. We’d already gotten a lot closer to GV’s spending targets over the last couple of years, but this update has accelerated our timeline for investing more in partnerships and advising other philanthropists. If you’re interested, please consider applying or referring candidates to lead our new partnerships function. And if you happen to be a philanthropist looking for advice on how to invest >$1M/year in new cause areas, please get in touch.
There are 8.1 billion people on the planet and afaict 8,099,999,999 of them donate less to my favorite causes & orgs than @Dustin Moskovitz. That was true before this update and it will remain true after it. Like everyone else I have elaborate views on how GV/OP should spend money/be structured etc but let the record also show that I appreciate the hell out of Dustin & Cari, we got so lucky 🥲
It's also noteworthy that Dustin/GV/OP is both by far the largest EA donor and the one EA folks most often single out to express frustration & disappointment about. I get why that is but you gotta laugh
I've never seen anyone express frustration or disappointment about Dustin, except for Habryka. However, Habryka seems to be frustrated with most people who fund anything / do anything that's not associated with Lightcone and its affiliates, so I don't know if he should count as expressing frustration at Dustin in particular.
Are you including things like criticizing OP for departing grantees too quickly and for not departing grantees quickly enough, or do you have something else in mind?
I view Dustin and OP as quite separate, especially before Holden's departure, so that might also explain our different experience.
There is a huge amount of work I am deeply grateful for that as far as I can tell is not "associated with Lightcone and its affiliates". Some examples:
- The historical impact of the Future of Humanity Institute
- The historical impact of MIRI
- Gwern's writing and work
- Joe Carlsmith's writing
- Basically all of Holden's intellectual output (even if I disagree with his leadership of OP and EA in a bunch of very important ways)
- Basically all of Owen Cotton Barratt's contributions (bar the somewhat obvious fuckups that came out last year, though I think they don't outshine his positive contributions)
- John Wentworth's contributions
- Ryan Greenblatt's and Buck Shlegeris's contributions
- Paul Christiano's and Mark Xu's research (I have disagreements with Paul on EA leadership and governance things, but I think his research overall has been great)
- Rohin Shah's many great cont
... (read more)I can confirm that Oliver dislikes us especially, and that other people dislike us as well.
("Disliking" feels a bit shallow, though I think a fair gloss. I have huge respect for a huge number of people at Open Philanthropy as well as many of the things you've done, in addition to many hard feelings and grievances.
It does sadly to me seem like we are in a world where getting many things right, but some things wrong, still can easily flip the sign on the impact of one's actions and make it possible to cause large amounts of harm, which is my feeling with regards to OP.
I feel confused what exact emotional and social relationship that should cause me to have with OP. I have similar feelings about e.g. many people at Anthropic. In many respects they seem so close to having an enormous positive impact, they think carefully through many important considerations, and try pretty hard to create an environment for good thinking, but in expected impact space so far away from where I wish they were.)
Apologies, again, for putting words in your mouth. I was using a little gallows humor to try to break the tension. It didn't work.
Lol, it did totally work and I quite liked your comment and laughed about it. I just wanted to clarify since IDK, it does all still feel a bit high-stakes and being clear seems valuable, but I think your comment was great and did indeed ease a bunch of tension in me.
Ok great. Well I just want to re-emphasize the distinction again between "OP" and the people who work at OP. It's not a homogenous blob of opinions, and AFAIK we didn't fire anybody related to this, so a lot of the individuals who work there definitely agree with you/want to keep working with you on things and disagree with me.
Based on your read of their feelings and beliefs, which I sincerely trust is superior to my own (I don't work out of the office or anything like that), there is empirically a chilling effect from my decisions. All I can say is that wasn't what I was aiming for, and I'll try to mitigate it if I can.
Thanks, I appreciate that. I might message you at random points in the coming months/years with chilling effects I notice (in as much as that won't exacerbate them), and maybe ideas to mitigate them.
I won't expect any response or much engagement, I am already very grateful for the bandwidth you've given me here, as one of the people in the world with the highest opportunity cost.
Sometimes I wish we had a laughing emoji here...
I'm pleasantly surprised at the DeepMind alignment team being the only industry team called out! I'm curious what you think we're doing right?
To be clear, I am concerned many of these things will get worse with the Deepming/Brain merger, and a lot of my datapoints are from before then, but I think the track record overall is still quite good.
Thanks for the update! Are there any plans to release the list of sub areas? I couldn't see it in this post or the blog post, and it seems quite valuable for other funders, small donors (like me!) and future grantees/org founders to know which areas might now be less well funded.
Here are the ones I know about: wild animal welfare (including averting human-caused harms), all invertebrate welfare (including farmed shrimp), digital minds, gene editing*.
I think this is close to all of them in the animal space. I believe there are also some things losing funding in other areas (e.g., see Habryka's comments), but I'm less familiar with that community.
*I don't know about gene editing for humans, like for malaria.
[edited to fix typo]
Good to know, what's the source of this info?
Edit: I retracted the below because I think it is unkind and wasn't truth-seeking enough. I apologise if I caused too much stress to @Dustin Moskovitz or @Alexander_Berger, even if I have disagreements with GVF/OP about things I very much appreciate what both of you are doing for the world, let alone 'EA' or its surrounding community.
Wait what, we're (or GV is) defunding animal stuff to focus more on AI stuff? That seems really bad to me, I feel like 'PR' damage to EA is much more coming from the 'AI eschaton' side than the 'help the animals' side (and also that interventions on animal welfare are plausibly much more valuable than AI)[1]e.g. here and here
No, the farm animal welfare budget is not changing, and some of the substreams GV are exiting (or not entering) are on the AI side. So any funding from substratgies that GV is no longer funding within FAW would be reallocated to other strategies within FAW (and as Dustin notes below, hopefully the strategies that GV will no longer fund can be taken forward by others).
Yep was about to ask the same question, it's such a glaring omission I imagine there might be reasons it has not been shared?
Even if there are it might be good to know what those reasons for not sharing are, otherwise it might draw even more attention.
I think some critiques of GVF/OP in this comments section could have been made more warmly and charitably.
The main funder of a movement's largest charitable foundation is spending hours seriously engaging with community members' critiques of this strategic update. For most movements, no such conversation would occur at all.
Some critics in the comments are practicing rationalist discussion norms (high decoupling & reasoning transparency) and wish OP's communications were more like that too. However, it seems there's a lot we don't know about what caused GFV/OP leadership to make this update. Dustin seems very concerned about GFV/OP's attack surface and conserving the bandwidth of their non-monetary resources. He's written at length about how he doesn't endorse rationalist-level decoupling as a rule of discourse. Given all of this, it's understandable that from Dustin's perspective, he has good reasons for not being as legible as he could be. Dishonest outside actors could quote statements or frame actions far more uncharitably than anything we'd see on the EA Forum.
Dustin is doing the best he can to balance between explaining his reasoning and adhering to legibility constraints ... (read more)
I generally agree with the spirit of empathy in this comment, but I also think you may be misinterpreting Dustin in a similar way to how others are. My understanding is that Dustin is not primarily driven by how other actors might use his funding / public comments against him. Instead, it is something like the following:
"Dustin doesn't want to be continually funding stuff that he doesn't endorse, because he thinks that doing things well and being responsible for the consequences of your actions is intrinsically important. He is a virtue ethicist and not a utilitarian in this regard. He feels that OP has funded things he doesn't endorse enough times in enough areas to not want to extend blanket trust, and thus feels more responsibility than before to evaluate cases himself, to make sure that both individual grants and higher-level funding strategies are aligned with his values. He believes in doing fewer things well than more things poorly, which is why some areas are being cut."
Obviously this could be wrong and I don't want Dustin to feel any obligation to confirm/not confirm it. I'm writing it because I'm fairly confident that it's at least more right than the prevailing narrative currently in the comments, and because the reasoning makes a fair amount of sense to me (and much more sense than the PR-based narrative that many are currently projecting).
(I don't want to spend too much time psychologizing here, though I do think a root cause analysis is useful, so I will comment a bit, but will bow out if this gets too much)
I feel like this doesn't really match with Dustin's other comments about repeatedly emphasizing non-financial concrete costs. I think Dustin's model is closer to "every time I fund something I don't endorse, I lose important forms of social capital, political capital, and feel a responsibility to defend and explain my choices towards others, for which I only have limited bandwidth and don't have time or energy for. As a result, I am restricting my giving to things that I feel excited to stand behind, which is a smaller set, and where I feel good about the tradeoffs of financial and non-financial costs".
Re: attack surface in my early comment, I actually meant attacks from EAs. People want to debate the borders, quite understandably. I have folks in my DMs as well as in the comments. Q: "Why did we not communicate more thoroughly on the forum"
A: "Because we've communicated on the forum before"
I don't think endorse vs. not endorse describes everything here, but it describes some if it. I do think I spend some energy on ~every cause area, and if I am lacking conviction, that is a harder expenditure from a resource I consider finite.
An example of a non-monetary cost where I have conviction: anxiety about potential retribution from our national political work. This is arguably not even EA (and not new), but it is a stressful side hustle we have this year. I had hoped it wouldn't be a recurring thing, but here we are.
An example of a non-monetary cost where I have less conviction: the opportunity cost of funding insect welfare instead of chicken, cow, or pig welfare. I think I could be convinced, but I haven't been yet and I've been thinking about it a long time! I'd much prefer to just see someone who actually feels strongly about that take the wheel. It is not a lot of $s in itself, bu... (read more)
That's brilliant Ariel and you articulate this far better and more forum-appropriate than my attempt at similar! I think in discussion whatever our own norms and preferences, we are likely to get more meaningful discussion if we bend at least somewhat towards those of the person we are talking with.
I find myself particularly disappointed in this as I was working for many years on projects that were intended to diversify the funding landscape, but Open Phil declined to fund those projects, and indeed discouraged me from working on them multiple times (most notably SFF and most recently Lightspeed Grants).
I think Open Phil could have done a much better job at using the freedom it had to create a diverse funding landscape, and I think Open Phil is largely responsible for the degree to which the current funding landscape is as centralized as it currently is.
I'm surprised to hear you say SFF and Lightspeed were trying to diversify the funding landscape, AND that it was bad that OpenPhil didn't fund them. My understanding was that there was already another donor (Jaan Tallinn) who wanted to make large donations, and you were trying to help them. To me, it seems natural for Jaan to fund these, and that this is great because it results in a genuinely independent donor. OpenPhil funding it feels more like a regranting program, and I don't see how that genuinely diversifies the landscape in the longterm (unless eg OpenPhil funded a longterm endowment for such a program that they can't later take away). Was the ask for them to fund the operations, or to add to the pool of money donated? Was the idea that, with more funding, these programs could be more successful and attract more mega donors from outside the community?
The point of Lightspeed Grants was explicitly to create a product that would allow additional funders (beyond Jaan) to distribute funding to important causes.
It also had the more immediate positive effect of increasing the diversity and impact of Jaan's funding, though that's not where I expected the big wins to come from and not my primary motivation for working on it. I still feel quite excited about this, but stopped working on it in substantial parts because Open Phil cut funding for all non-LW programs to Lightcone.
The ask here was for development cost and operations, not for any regranting money.
Basically.
To be clear, I think it's not a crazy decision for Open Phil to think that Jaan is in a better position to fund our SFF and Lightspeed work (though not funding us for Lightspeed did have a pretty substantial effect on our ability to work on it). The bigger effect came from both the implicit and explicit discouragement of working on SFF and Lightspeed over the years, mostly picked up from random conversations with Open Phil staff and peop... (read more)
I feel quite aligned with you spiritually on this topic: we both want more diversified funding for these causes, and see Good Ventures-as-sole-funder both getting in the way of that outcome and creating a variety of second order problems. I'm sorry to hear that OP may have actively dissuaded you from pursuing diversification. Folks like you, as well as OP staff, will now spend more of their time pursuing other funding sources, and I expect that to be strongly positive for the causes and the general EA project over the long run.
In the short run, I agree we have recently lost valuable time and energy via this path, and I regret that and accept a good deal of responsibility. When FTX showed up on the scene, it felt like plurality was happening, and as soon as they went away, I became quite fixated on this topic again.
Thanks for your thoughts, Dustin. I think it was a mistake at the time - and I said as much - to think that FTX and OpenPhil represented sufficient plurality. But I definitely didn't think FTX would blow up as it did and given that people can only do so many things, it's understandable that people didn't focus enough on donor diversification.
I didn't think of it as sufficient, but I did think of it as momentum. "Tomorrow, there will be more of us" doesn't feel true anymore.
Is it possible to elaborate on how certain grants but not others would unusually draw on GV's bandwidth? For example, what is it about digital minds work that draws so much more bandwidth than technical AI safety grants? Personally I find that this explanation doesn't actually make any sense as offered without more detail.
I think the key quote from the original article is "In the near term, we want to concentrate our giving on a more manageable number of strategies on which our leaders feel more bought in and with which they have sufficient time and energy to engage." Why doesn't Good Ventures just want to own the fact that they're just not bought in on some of these grant areas? "Using up limited capacity" feels like a euphemism.
My best guess at the situation is that the "limited capacity" is a euphemism for "Dustin/Cari don't think these things are valuable and Open Phil doesn't have time to convince them of their value".
Separately, my guess is one of the key dimensions on which Dustin/Cari have strong opinions here are things that affect Dustin and Cari's public reputation in an adverse way, or are generally "weird" in a way that might impose more costs on Dustin and Cari.
I generally believe they are all valuable (in expectation anyway).
Your second statement is basically right, though my personal view is they impose costs on the movement/EA brand and not just us personally. Digital minds work, for example, primes the idea that our AI safety concerns are focused on consciousness-driven catalysts ("Terminator scenarios"), when in reality that is just one of a wide variety of ways AI can result in catastrophe.[What I thought I was affirming here seems to be universally misunderstood by readers, so I'm taking it back.]
I hope to see everything funded by a more diverse group of actors, so that their dollar and non-dollar costs are more distributed. Per my other comment, I believe you (Oliver) want that too.
I would be very surprised if digital minds work of all things would end up PR-costly in relevant ways. Indeed, my sense is many of the "weird" things that you made a call to defund form the heart of the intellectual community that is responsible for the vast majority of impact of this ecosystem, and I expect will continue to be the attractor for both funding and talent to many of the world's most important priorities.
An EA community that does not consider whether the minds we aim to control have moral value seems to me like one that has pretty seriously lost its path. Not doing so because some people will walk away with a very shallow understanding of "consciousness" does not seem to me like a good reason to not do that work.
I think you absolutely have a right to care and value your personal reputation, but I do not think your judgement of what would hurt the "movement/EA brand" is remotely accurate here.
I think something worth noting is that most (all?) the negative PR on EA over the last year has focused on areas that will continue to be funded. The areas that were cut for the most part have, to my knowledge, not drawn negative media coverage (maybe Wytham Abbey is an exception — not sure if the sale was related to this though).
Of course, these areas could still be PR risks, and the other areas could be worth funding in spite of the PR risks.
Edit: For disagree voters, I'm curious why you disagree? A quick Google of the negative coverage of OpenPhil or EA all appear to be areas that OpenPhil has not pulled out of, at least to my knowledge. I'm not arguing that they shouldn't have made this determination, but I'd be interested in counter-examples if you disagree (negative media coverage of EA work in an area they are not granting in any more). I'm sure there are some, but my read is that most the negative media is covering work they are still doing. I see some minor off hand remarks about digital sentience, but negative media is overwhelmingly focused on AI x-risk work or FTX/billionaire philanthropy.
Yes, there will definitely still be a lot of negative attention. I come from the "less is less" school of PR.
Can you say more about that? You think our prior actions caused additional funding from Vitalik, Jed, and Jaan?
We’re still going to be funding a lot of weird things. I just think we got to a place where the capital felt ~infinite and we assumed all the other inputs were ~infinite too. AI safety feels like it deserves more of those resources from us, specifically, in this period of time. I sincerely hope it doesn’t always feel that way.
I don't know the full list of sub-areas, so I cannot speak with confidence, but the ones that I have seen defunded so far seem to me like the kind of things that attracted Jed, Vitalik and Jaan. I expect their absence will atrophy the degree to which the world's most ethical and smartest people want to be involved with things.
To be more concrete, I think funding organizations like the Future of Humanity Institute, LessWrong, MIRI, Joe Carlsmith, as well as open investigation of extremely neglected questions like wild animal suffering, invertebrate suffering, decision theory, mind uploads, and macrostrategy research (like the extremely influential Eternity in Six Hours paper) played major roles in people like Jaan, Jed and Vitalik directing resources towards things I consider extremely important, and Open Phil has at points in the past successfully supported those programs, to great positive effect.
In as much as the other resources you are hoping for are things like:
- more serious consideration from the world's smartest people,
- or more people respecting the integrity and honesty of the people working on AI Safety,
- or the ability for people to successfully coordin
... (read more)It is the case that we are reducing surface area. You have a low opinion of our integrity, but I don't think we have a history of lying as you seem to be implying here. I'm trying to pick my battles more, since I feel we picked too many. In pulling back, we focused on the places somewhere in the intersection of low conviction + highest pain potential (again, beyond "reputational risks", which narrows the mind too much on what is going on here).
>> In general, I think people value intellectual integrity a lot, and value standing up for one's values. Building communities that can navigate extremely complicated domains requires people to be able to follow arguments to their conclusions wherever that may lead, which over the course of one's intellectual career practically always means many places that are socially shunned or taboo or reputationally costly in the way that seems to me to be at the core of these changes.
I agree with the way this is written spiritually, and not with the way it is practiced. I wrote more about this here. If the rationality community wants carte blanche in how they spend money, they should align with funders who sincerely believe more in the specific im... (read more)
And to get a little meta, it seems worth pointing out that you could be taking this whole episode as an empirical update about how attractive these ideas and actions are to constituents you might care about and instead your conclusion is "no, it is the constituents who are wrong!"
>> Let Open Philanthropy decide whether they think what we are doing helps with AI risk, or evaluate it yourself if you have the time.
Indeed, if I have the time is precisely the problem. I can't know everyone in this community, and I've disagreed with the specific outcomes on too many occasions to trust by default. We started by trying to take a scalpel to the problem, and I could not tie initial impressions at grant time to those outcomes well enough to feel that was a good solution. Empirically, I don't sufficiently trust OPs judgement either.
There is no objective "view from EA" that I'm standing against as much as people portray it that way here; just a complex jumble of opinions and path dependence and personalities with all kinds of flaws.
>> Also, to be clear, my current (admittedly very limited sense) of your implementation, is that it is more of a blacklist than a simple redirecting of resources towards fewer priority areas.
So with that in mind this is the statement that felt like an accusation of lying (not an accusation of a history of lying), and I think we have arrived at the reconciliation that doesn't involve lying: broad strokes were pragmatically needed i... (read more)
(Just want to say, I really appreciate you sharing your thoughts and being so candid, Dustin. I find it very interesting and insightful to learn more about your perspective.)
Though to be clear, I think an important belief of mine, which I am confident the vast majority of readers here will disagree with me, is that the aggregate portfolio of Open Phil and Good Ventures is quite bad for the world (especially now, given the updated portfolio).
As such, it's unclear to me what I should feel about a change where some of the things I've done are less appealing to you. You are clearly smart and care a lot about the same things as I care about, but I also genuinely think you are causing pretty huge harm for the world. I don't want to alienate you or others, and I would really like to maintain good trade relationships in as much as that is possible, since we we clearly have identified very similar crucial levers in the world, and I do not want to spend our resources in negative-sum conflict.
I still think hearing that the kind of integrity I try to champion and care about did fail to resonate with you, and failed to compel you to take better actions in the world, is crucial evidence that I care a lot about. You clearly are smart and thoughtful about these topics and I care a lot about the effect of my actions on people like you.
(This comment overall isn't obviously immediately relevant, and probably isn't worth responding to, but I felt bad having my previous comment up without giving this important piece of context on my beliefs)
Can you elaborate on this? Your previous comments explain why you think OP's portfolio is suboptimal, but not why you think it is actively harmful. It sounds like you may have written about this elsewhere.
My experience of reading this thread is that it feels like I am missing essential context. Many of the comments seem to be responding to arguments made in previous, perhaps private, conversations. Your view that OP is harmful might not be immediately relevant here, but I think it would help me understand where you are coming from. My prior (which is in line with your prediction that the vast majority of readers would disagree with your comment) is that OP is very good.
Sorry, maybe I missed something, where did I imply you have a history of lying? I don't currently believe that Open Phil or you have a history of lying. I think we have disagreements on dimensions of integrity beyond that, but I think we both care deeply about not lying.
I don't really know what you mean by this. I don't want carte blanche in how I spend money. I just want to be evaluated on my impact on actual AI risk, which is a priority we both share. You don't have to approve of everything I do, and indeed think allowing people to choose their means by which to achieve a long-term goal, is one of the biggest reasons for historical EA philanthropic success (as well as a lot of the best parts of Silicon Valley).
A complete blacklist of a whole community seems extreme, and rare, even for non-EA philanthropists. Let Open Philanthropy decide whether they think... (read more)
My view is that rationalists are the force that actively makes room for it (via decoupling norms), even in "guest" spaces. There is another post on the forum from last week that seems like a frankly stark example.
I cannot control what the EA community chooses for itself norm-wise, but I can control whether I fuel it.
Anyone know what post Dustin was referring to? EDIT: as per a DM, probably this one.
I didn't mean to argue against the Digital Minds work here and explicitly see them as within the circle of moral concern. However, I believe that a different funder within the EA community would still mitigate the costs I'm talking about tactically. By bringing up the topic there, I only meant to say this isn't all about personal/selfish ends from my POV (*not* that I think it nets out to being bad to do).
It is not a lot of money to fund at this stage as I understand it, and I hope to see if funded by someone who will also engage with the intellectual and comms work. For GV, I feel more than fully occupied with AI Safety.
Appreciate you engaging thoughtfully with these questions!
I'm slightly confused about this specific point - it seems like you're saying that work on digital minds (for example) might impose PR costs on the whole movement, and that you hope another funder might have the capacity to fund this while also paying a lot of attention to the public perception.
But my guess is that other funders might actually be less cautious about the PR of the whole movement, and less invested in comms that don't blow back on (for example) AI safety.
Like, personally I am in favour of funder diversity but it seems like one of the main things you lose as things get more decentralised is the ability to limit the support that goes to things that might blow back on the movement. To my taste at least, one of the big costs of FTX was the rapid flow of funding into things that looked (and imo were) pretty bad in a way that has indirectly made EA and OP look bad. Similarly, even if OP doesn't fund things like Lighthaven for maybe-optics-ish reasons, it still gets described in news articles as an EA venue.
Basically, I think better PR seems good, and more funding diversity seems good, but I don't expect the movement is actually going to get both?
(I do buy that the PR cost will be more diffused across funders though, and that seems good, and in particular I can see a case for preserving GV as something that both is and seems reasonable and sane, I just don't expect this to be true of the whole movement)
“PR risk” is an unnecessarily narrow mental frame for why we’re focusing.
Risky things are risky in multiple ways. Diffusing across funders mitigates some of them, some of the time.
AND there are other bandwidth issues: energy, attention, stress, political influence. Those are more finite than capital.
For what it's worth, as a minor point, the animal welfare issues I think are most important, and the interventions I suspect are the most cost-effective right now (e.g. shrimp stunning), are basically only fundable because of EA being weird in the past and willing to explore strange ideas. I think some of this does entail genuine PR risk in certain ways, but I don't think we would have gotten most of the most valuable progress that EA has made for animal welfare if we paid attention to PR between 2010 and 2021, and the animal welfare space would be much worse off. That doesn't mean PR shouldn't be a consideration now, but as a historical matter, I think it is correct that impact in the animal space has largely been driven by comfort with weird ideas. I think the new funding environment is likely a lot worse for making meaningful progress on the most important animal welfare issues.
The "non-weird" animal welfare ideas that are funded right now... (read more)
I don't think these areas have run into PR issues historically, but they are perceived as PR risks.
I agree that I make two separate points. I think evaluating digital sentience seems pretty important from a "try to be a moral person" perspective, and separately, I think it's just a very reasonable and straightforward question to ask that I expect smart people to be interested in and where smart people will understand why someone might want to do research on this question. Like, sure, you can frame everything in some horribly distorting way, and find some insult that's vaguely associated with that framing, but I don't think that's very predictive of actual reputational risk.
Most of the sub-cause areas that I know about that have been defunded are animal welfare priorities. Things like insect suffering and wild animal welfare are two of the sub-cause areas that are getting defunded, which I both considered to be among the more important animal welfare priorities (due to their extreme neglectedness). I am not being dismissive of either global health or animal welfare people, they are being affected by this just as much (I know less about global health, and my sense is the impact of these changes are less bad there, but I still expect a huge negative chilling effect on people trying to think carefully about the issues around global health).
Yes, I'm explicitly pro-funding by others. Framing the costs as "PR" limits the way people think about mitigating costs. It's not just "lower risk" but more shared responsibility and energy to engage with decision making, persuading, defending, etc.
@Dustin Moskovitz I think some of the confusion is resulting from this:
In my reading of the thread, you first said "yeah, basically I think a lot of these funding changes are based on reputational risk to me and to the broader EA movement."
Then, people started challenging things like "how much should reputational risk to the EA movement matter and what really are the second-order effects of things like digital minds research."
Then, I was expecting you to just say something like "yeah, we probably disagree on the importance of reputation and second-order effects."
But instead, it feels (to me) like you kind of backtracked and said "no actually, it's not really about reputation. It's more about limited capacity– we have finite energy, attention, stress, etc. Also shared responsibility."
It's plausible that I'm misun... (read more)
>> In my reading of the thread, you first said "yeah, basically I think a lot of these funding changes are based on reputational risk to me and to the broader EA movement."
I agree people are paraphrasing me like this. Let's go back to the quote I affirmed: "Separately, my guess is one of the key dimensions on which Dustin/Cari have strong opinions here are things that affect Dustin and Cari's public reputation in an adverse way, or are generally "weird" in a way that might impose more costs on Dustin and Cari."
I read the part after "or" as extending the frame beyond reputation risks, and I was pleased to see that and chose to engage with it. The example in my comment is not about reputation. Later comments from Oliver seem to imply he really did mean just PR risk so I was wrong to affirm this.
If you look at my comments here and in my post, I've elaborated on other issues quite a few times and people keep ignoring those comments and projecting "PR risk" on to everything.
I feel incapable of being heard correctly at this point, so I guess it was a mistake to speak up at all and I'm going to stop now.[Sorry I got frustrated; everyone is trying their best to do the most good here] I would appreciate if people did not paraphrase me from these comments and instead used actual quotes.I want to echo the other replies here, and thank you for how much you've already engaged on this post, although I can see why you want to stop now.
I did in fact round off what you were saying as being about PR risk yesterday, and I commented as such, and you replied to correct that, and I found that really helpful - I'm guessing a lot of others did too. I suppose if I had already understood, I wouldn't have commented.
At the risk of overstepping or stating the obvious:
It seems to me like there's been less legibility lately, and I think that means that a lot more confusion brews under the surface. So more stuff boils up when there is actually an outlet.
That's definitely not your responsibility, and it's particularly awkward if you end up taking the brunt of it by actually stepping forward to engage. But from my perspective, you engaging here has been good in most regards, with the notable exception that it might have left you more wary to engage in future.
Ah, gotcha. This makes sense– thanks for the clarification.
I've looked over the comments here a few times, and I suspect you might think you're coming off more clearly than you actually are. It's plausible to me that since you have all the context of your decision-making, you don't see when you're saying things that would genuinely confuse others.
For example, even in statement you affirmed, I see how if one is paying attention to the "or", one could see you technically only/primarily endorsing the non-PR part of the phrase.
But in general, I think it's pretty reasonable and expected that people ended up focusing on the PR part.
More broadly, I think some of your statements have been kind of short and able to be interpreted in many ways. EG, I don't get a clear sense of what you mean by this:
... (read more)Sorry to hear that, several people I've spoken to about this offline also feel that you are being open and agreeable and the discussion reads from the outside as fairly civil, so except perhaps with the potential heat of this exchange with Ollie, I'd say most people get it and are happy you participated, particularly given that you didn't need to. For myself, the bulk of my concern is with how I perceive OP to have handled this given their place in the EA community, rather than my personal and irrelevant partial disagreement with your personal funding decisions.
[edited to add "partial" in the last sentence]
I wonder if this is an argument for Good Ventures funding more endowments. If they endow a re-granter that funds something weird, they can say "well the whole point of this endowment was to diversify decision-making; it's out of our hands at this point". From the perspective of increasing the diversity of the funding landscape, a no-strings endowment seems best, although it could have other disadvantages.
[By 'endowment' I'm suggesting a large, one-time lump sum given to a legally independent organization. That organization could choose to give away the endowment quickly and then dissolve, or give some legally mandated minimum disbursement every year, or anything in between.]
>> If they endow a re-granter that funds something weird, they can say "well the whole point of this endowment was to diversify decision-making; it's out of our hands at this point".
I proposed this myself at one point, and the team politely and quite correctly me informed me that projecting this response from critics was naive. We are ultimately responsible for the grants downstream of our decisions in the eyes of the world, regardless of who made intermediate decisions.
As an example of how this has played out in practice, we're known (and largely reviled) locally in San Francisco for supporting the controversial DA Chesa Boudin. In fact, I did not even vote for him (and did not vote in the recall), and the only association is the fact that we made a grant in 2019 to a group that supported him in later years, for totally unrelated work. We made a statement to clarify all this, which helped a little on the margin, but did not substantially change the narrative.
Like I said, I proposed it myself. So I'm sympathetic to the idea, and maybe we'll come back to it in some years if it truly becomes impossible to achieve real plurality.
Yes, I think funding an endowment would have been a great thing to do (and something I advocated for many times over the years). My sense is that it's too late now.
(Mostly agreeing)
I feel like:
1. I haven't seen any writing about how disagreements between Dustin/Cari, and other OP execs, have changed priorities. (Or how other "political" considerations changed priorities)
2. I'm sure that there have been disagreements between them, that have changed priorities.
3. I would naively expect many organizations to swipe "changes made, just because some other exec wanted them" under the rug of some other reason or other, like, "Well, we have a lot of uncertainty on this topic." Likewise, I don't trust the reasons that many organizations give to many of their non-obvious high-level decisions.
Therefore, I think it's pretty natural to conclude that there's probably something funky going on, as I'd similar expect for other institutions. That some/many of the reasons for high-level decisions are political instead of epistemic.
I'd similarly assume that many high-level people at OP would like to signal these differences, but it would be difficult for them to do so (as is usually the case), so wouldn't mind EAs making conclusions like this.
That said - if it is the case that there are important political reasons for things, I think it would be reall... (read more)
I wrote at length about my views on epistemic confidence here https://medium.com/@moskov/works-in-progress-the-long-journey-to-doing-good-better-9dfb68e50868
Kudos for commenting here, and in the rest of this thread!
Just fyi, I find your comments in threads like these a lot more informative than blog posts like the one you linked to.
I think that blog post is reasonable, but it is fairly high-level, and I find that the devil is typically in the details. I feel like I've seen other people, both good and bad, post high-level epistemic takes that seemed good to me, so I'm just not how much I can take away from posts like that specifically.
But comments to questions explaining specific decisions is something I find quite useful!
I'm not detailing specific decisions for the same reason I want to invest in fewer focus areas: additional information is used as additional attack surface area. The attitude in EA communities is "give an inch, fight a mile". So I'll choose to be less legible instead.
As a datapoint (which you can completely ignore), I feel like in the circles I travel in, I've heard a lot more criticism of OP that look more like "shady non-transparent group that makes huge decisions/mistakes without consulting anyone except a few Trusted People who all share the same opinions."
There are certainly some cases in which the attack surface is increased when you're fully open/transparent about reasoning.
But I do think it can be easy to underestimate the amount of reputational damage that OP (and you, by extension) take from being less legible/transparent. I think there's a serious risk that many subgroups in EA will continue to feel more critical of OP as it becomes more clear that OP is not interested in explaining its reasoning to the broader community, becomes more insular, etc. I also suspect this will have a meaningful effect on how OP is perceived in non-EA circles. I don't mean e/accs being like "OP are evil doomers who want to give our future to China"– I mean neutral third-parties who dispassionately try to form an impression of OP. When they encounter a... (read more)
Thanks for explaining your position here!
(Again, feel free to stop this at any time)
> The attitude in EA communities is "give an inch, fight a mile". So I'll choose to be less legible instead.
I assume you meant that EAs are the ones fighting OP, with things like poor comments?[1]
If you feel that way, that seems intensely bad, for both you and the rest of the community.
I'm obviously saddened that reactions from this community seem to have been so frustrating. It also generally seems unhealthy to have an epistemic environment like this.
I'm really curious about ways to make these communications go better. My impression is that:
1. Communication is very important.
2. Most (though not all) people in OP+EA are very reasonable and altruistic.
3. It is often the case that posting important content on the EA Forum is unpleasant. Sometimes the most upset people might respond, for instance, and upvotes/downvotes can be scary. I know a lot of people who very much avoid this.
This doesn't seem like it should be an impossible knot to me. Even if it cost $10M+ to hire some professional business coaches or something, that seems very likely worth it to me. We could move key conversations to an... (read more)
[Edit: I'm not an expert in digital minds at all, don't take too much from this!]
Minor point, but I'm still really not convinced by research into digital minds. My personal model is that most of that can be figured out post-AGI, and we first should model epistemic-lock-in, before we are worried about it in specific scenarios. Would be happy to see writing explaining the expected value! (That said, I'm not really making decisions in this area, just a passing observer)
Update: I see above that Oliver seems to really be bought into it, which I didn't realize. I would flag that I'm very uncertain here - there definitely could be some strong reasons I'm not aware of.
Also, this isn't "me not liking weird things" - I'm a big fan of weird research, just not all weird research (and I'm not even sure this area seems that weird now)
Hi Ozzie,
One could also have argued for figuring out farmed animal welfare after cheap animal food (produced in factory-farms) is widely available? Now that lots of people are eating factory-farmed animals, it is harder to role back factory-farming.
I want to express a few things.
First, empathy, for:
Second, a little local disappointment in OP. At some point in the past it seemed to me like OP was trying pretty hard to be very straightforward and honest. I no longer get that vibe from OP's public comms; they seem more like they're being carefully crafted for something like looking-good or being-defensible while only saying true things. Of course I don’t know all the constraints they’re under so I can’t be sure this is a mistake. But I personally feel a bit sad about it — I think it makes it harder for people to make useful updates from things OP says, which is awkward because I think a bunch of people kind of look to OP for leadership. I don’t think anything is crucially wrong here, but I’m worried about people missing the upside from franker communication, and... (read more)
Just flagging that I think "OP [is] open to funding XYZ areas if a new funder appears who wants to partner with them to do so" accurately describes the status quo. In the post above we (twice!) invited outreach from other funders interested in the some of these spaces, and we're planning to do a lot more work to try to find other funders for some of this work in the coming months.
I am skeptical that a new large philanthropist would be well-advised by doing their grantmaking via OP (though I do think OP has a huge amount of knowledge and skill as a grantmaker). At least given my current model, it seems hard to avoid continuing conflict about the shared brand and indirect-effects of OP on Good Ventures.
I think any new donor, especially one that is smaller than GV (as is almost guaranteed to the the case), would end up still having their donations affect Good Ventures and my best understanding of the things Dustin is hoping to protect via this change.
If Dustin can't communicate that he doesn't endorse every aspect of, or can't take responsibility for, everything that is currently funded through OP, I doubt that the difficult-to-track "from which source did OP spend this money" aspect of an additional donor would successfully avoid the relevant conflicts.
My best guess is that if OP attracts an additional large donor above ~$200M, it seems best for some people to leave OP, establish a new organization that the now donor would actually be confident will be independent from Dustin's preferences, while maintaining a collaborative relationship with OP to share thoughts and insights.
I am not super confident of this, but I don't see a clear line that would allow a new donor to have confidence that OP staff isn't still going to heavily take into account Dustin's reputation and non-financial priorities in the recommendations to the new donor.
That could well be, but my experience was having another foundation, like FTX, didn't insulate me from reputation risks either. I'm just another "adherent of SBF's worldview" to outsiders.
I'd like to see a future OP that is not synonymous with GVF, because we're just one of the important donors instead of THE important donor, and having a division of focus areas currently seems viable to me. If other donors don't agree or if staff behaves as if it isn't true, then of course it won't happen.
My hope is that having other donors for OP would genuinely create governance independence as my apparent power comes from not having alternate funding sources*, not from structural control. Consequently, you and others lay blame on me even for the things we don't do. I would be happy to leave the board even, and happy to expand it to diminish my (non-controlling) vote further. I did not want to create a GVF hegemony any more than you wanted one to exist. (If the future is a bunch of different orgs, or some particular "pure" org, that's good by me too; I don't care about OP aggregating the donors if others don't see that as useful.)
But I do want agency over our grants. As much as the whole debate has been framed (by everyone else) as reputation risk, I care about where I believe my responsibility lies, and where the money comes from has mattered. I don't want to wake up anymore to somebody I personally loathe getting platformed only to discover I paid for the platform. That fact matters to me.
* Notably just for the "weird" stuff. We do successfully partner with other donors now! I don't get in their way at all, as far as I know.
Just chiming in to have more than Habryka's view represented here. I think it's not unreasonable in principle to think that OP and GV could create PR distance between themselves. I think it will just take time, and Habryka is being moderately too pessimistic (or, accurately pessimistic in the short term but not considering reasonable long-term potential). I'd guess many think-tank type organizations have launched on the strength of a single funder and come to advise many funders, having a distinct reputation from them -- OP seems to be pursuing this more strongly now than in the past, and while it will take time to work, it seems like a reasonable theory of change.
I have updated, based on this exchange, that OP is more open to investment in non-GV-supported activities than previously came through. I'm happy to hear that.
This is eminently reasonable. Any opposition to these changes I've aired here is more about disappointment in some of the specific cause areas being dropped and the sense that OP couldn't continue on them without GV; I'm definitely not intending to complain about GV's decision, and overall I think OP attempting to diversify funding sources (and EA trying to do so at well) is very, very healthy and needed.
Thanks, I am glad that you are willing to do this and am somewhat relieved that your perspective on this seems more amenable to other people participating with stuff on their own terms than I was worried about. I am still concerned, and think it's unlikely that other donors for OP would be capable of getting the appropriate levels of trust with OP for this to work (and am also more broadly quite worried about chilling effects of many types here), but I do genuinely believe you are trying to mitigate those damages and see many of the same costs that I see.
... (read more)I've long taken for granted that I am not going to live in integrity with your values and the actions you think are best for the world. I'm only trying to get back into integrity with my own.
OP is not an abstraction, of course, and I hope you continue talking to the individuals you know and have known there.
To clarify, I did see the invitations to other funders. However, my perception was that those are invitations to find people to hand things off to, rather than to be a continuing partner like with GV. Perhaps I misunderstood.
I also want to be clear that the status quo you're articulating here does not match what I've heard from former grantees about how able OP staff are to participate in efforts to attract additional funding. Perhaps there has been quite a serious miscommunication.
+1 to Alexander's POV
I'm confused about what you're saying here. It feels like maybe you're conflating OP with GV? (Which may be functionally a reasonable approximation 99% of the time, but gets in the way of the point of the conversation here in the 1%.) e.g. at one point you say "better if OP doesn't have to spend its own money", but as far as I understand things, to a first approximation Open Philanthropy doesn't have any money; rather, Good Ventures does.
In the opening post, Alexander invites people interested in funding research on the potential moral patienthood of digital minds to reach out to OP, which I took as indicative of continued interest.
This isn’t true for any of the other sub-focus areas that will be exited though, which I thought was strange. Given that nothing other than digital minds work was listed, how would any potential donors or people who know potential donors know about OP exiting things like invertebrates or wild animal welfare?
(I’m basing those sub-focus areas on the comments in this thread because of exactly this problem - I’m unclear if there’s more being exited or if those two are definitely part of it)
I assume that basically all senior EAs agree, and that they disprefer the current situation.
I'd flag though that I'm really not sure how well-equipped these nonprofits are to change this, especially without more dedicated resources.
Most nonprofit projects I see in EA, seem to be funded approximately at-cost. An organization would get a $200k grant to do work, that costs them approximately $200k to do.
Compare this to fast-growing businesses. These businesses often have significant sales and marketing budgets, sometimes this makes up 20-60% of the cost of the business. This is how these businesses expand. If these businesses charged at-cost, they would never grow.
I feel like it's assumed that the nonprofits have the responsibility to find ways of growing, despite them not getting much money from donors to actually do so. Maybe it's assumed that they'll do this in their spare time, and that it will be very easy?
It seems very reasonable to me that if we think growth is possible and valuable, potentially 20-40% of OP money ... (read more)
I agree but I want to be clear that I don't think senior EAs are innocent here. I agree with Habryka that this is a situation that was made by a lot of the senior EAs themselves who actively went all in on only two funders (now down to one) and discouraged a lot of attempts to diversify philanthropy.
I encouraged people against earning to give before (though I updated sharply after 2022), and I largely regret that move. (I don't think of myself as senior, especially at the time, but I'm unusually vocal online so I wouldn't be surprised if I had a disproportionate influence).
I'm not thinking just discouraging attempts to diversify funding of one's own org, but also discouraging earning to give, discouraging projects to bring in more donors, etc.
Yea, that seems bad. It felt like there was a big push a few years ago to make a huge AI safety researcher pipeline, and now I'm nervous that we don't actually have the funding to handle all of that pipeline, for example.
For sure. Not only the lack of funding to handle the pipeline, but there seems to be increasing concern around the benefits to harm tradeoff of technical AI research too.
Perhaps like with the heavy correction against earning to give a few years ago which now seems likely a mistake, maybe theres a lesson to be learned against overcorrecting against the status quo in any direction too quickly...
I also hope this doesn't come off as me being upset with OP / EA Funders. I think they're overall great (compared to most alternatives), but also that they are very important - so when they do get something wrong, that's a big deal and we should think through it.
Yeah, Cari and Dustin especially are a large part of what made a lot of this ecosystem possible in the first place, they seem like sincere people, and ultimately it's "their money" and their choice to do with their money what they genuinely believe can achieve the most good.
I generally agree, but with reservations.
- I think that Cari and Dustin's funding has obviously created a lot of value. Maybe, even ~60% of all shapley EA value.
- I personally don't feel like I know much of what Cari and Dustin actually believe about things, other than that they funded OP. They both seem to have been fairly private.
- At this point, I'm hesitant to trust any authority figure that much. "Billionaires in tech, focused on AI safety issues" currently has a disappointingly mixed track record.
- 'ultimately it's "their money" and their choice to do with their money what they genuinely believe can achieve the most good. ' -> Sure, but this is also true of
- "genuinely believe can achieve the most good" -> Small point, but I'm sure some of this is political. "Achieve the most good" often means, "makes the funder look better, arguably so that they could do more good later on". Some funders pay f
... (read more)allmany[1] big donors. I think that all big donors should probably get more evaluation and criticism. I'm not sure how many specific actions to take differently when knowing "it's their choice to do with their money what they genuinely believe can achieve the most good."Thinking about this more:
I think there's some frame behind this, like,
"There are good guys and bad guys. We need to continue to make it clear that the good guys are good, and should be reluctant to draw attention to their downsides."
In contrast, I try to think about it more like,
"There's a bunch of humans, doing human things with human motivations. Some wind up producing more value than others. There's expected value to be had by understanding that value, and understanding the positives/negatives of the most important human institutions. Often more attention should be spent in trying to find mistakes being made, than in highlighting things going well."
Thanks for elucidating your thoughts more here.
I agree this will be a (very) bad epistemic move. One thing I want to avoid is disincentivizing broadly good moves because their costs are more obvious/sharp to us. There are of course genuinely good reasons to criticize mostly good but flawed decisions (people like that are more amenable to criticism so criticism of them is more useful, their decisions are more consequential). And of course there are alternative framings where critical feedback is more clearly a gift, which I would want us to move more towards.
That said, all of this is hard to navigate well in practice.
Looking through some of these comment threads, I wonder if we're being perhaps a little too harsh and expecting a little too much of our funders at times. Being a funder is sacrificial and stressful. When I see comments from @Dustin Moskovitz like the ones below, I wonder if it might be better if we were a little more gentle and a little more kind in the way we have these discussions.
"I'm not detailing specific decisions for the same reason I want to invest in fewer focus areas: additional information is used as additional attack surface area. The attitude in EA communities is "give an inch, fight a mile". So I'll choose to be less legible instead.""
"It is the case that we are reducing surface area. You have a low opinion of our integrity, but I don't think we have a history of lying as you seem to be implying here."
"Indeed, if I have the time is precisely the problem. I can't know everyone in this community, and I've disagreed with the specific outcomes on too many occasions to trust by default."
"AND there are other bandwidth issues: energy, attention, stress, political influence. Those are more finite than capital."
But I could well be wrong and the debate could just be very robust and completely fine. Unsure...
Is more information available about the process and timetable of GV transitioning out of these subareas? Some of word choice here (surprise, not better anticipating) makes it sound like the transition may be too quick.
Part of being a responsible funder working toward fostering a healthy ecosystem is not pulling the rug out on departing grantees too quickly -- especially where the change in direction has little or nothing to do with the merits of the grantees' work, and where the funder's departure will have significant systemic effects on a subarea. To be clear, I don't think a funder is obliged to treat its prior funding decisions as some sort of binding precedent. But the funder's prior actions will have often contributed to the pickle in which the subarea's actors find themselves, and that usually calls for grace and patience during the exit transition.
I might analogize to the rules for laying off workers in the US; we expect more notice when there is a mass layoff. It makes sense that because the presence of a lot of new jobseekers in the same line of work and geographic area makes life harder for those who lost their job in a mass layoff vs. a isolated/small layoff. So it likely is with grantees as well.
Are OP staff planning to make any comments about grants they would counterfactually have recommended to GV, without this policy change?
My guess is that myself and other donors would potentially be interested in funding these.
Am I right in saying that OP will no longer be able to do what it thinks is most effective in some sense? That seems important to note.
An alternative theory is that OP doesn't think these cause areas are the most effective one but wants to create new buckets and has been told 'no' which seems less notable.
I think this is a complicated question - it's always been the case that individual OP staff had to submit grants to an overall review process and were not totally unilateral decision makers. As I said in my post above, they (and I) will now face somewhat more constraints. I think staff would differ in terms of how costly they would assess the new constraints as being. But it's true this was a GV rather than OP decision; it wasn't a place where GV was deferring to OP to weigh the costs and benefits.
FWIW I certainly agree with "non-trivial"; "huge" is a judgment call IMO. We'll see!