Doing EA Better: Preamble, Summary, and Introduction

Minor complaint / disagreement:

Norms must be enforced consistently, applying to senior EAs just as much as newcomers.

I think this is unreasonable, in the other direction. Newcomers (and outsiders) can and should be reminded or informed about norm violations the first time or times they screw up, and it should certainly take more than one time before there is any censure. Senior EAs should get far less flexibility and understanding. And I'm not blameless - there are at least two times that can recall I was rebuked for things privately, and I think it was good, and plausibly should have been more stringent and/or public; I have little excuse.

This is especially unfortunate because the level of discourse we aspire to, in my view, needs to be even better than what is already normal here. And many of the things that improvement entails are the points being made in this post! (And that's true even though the current norms are far better than what is seen elsewhere - the failures that the current post suggests correcting are already fairly egregious compared to my current expectations.) So I, at least, can say I hope to be held to a higher standard, and am happy to be told if and when I am failing to do so.

Michael_PJ

and plausibly should have been more stringent and/or public

Just wanted to comment on the public aspect. I think in general the old maxim remains true: "praise in public, criticise in private". I think it's generally better when people receive feedback about norm violations privately. I also think it's likely that this does happen today, and that does mean that it's hard to know how much norm violation policing is going on, because much of it may be private.

* https://www.currentaffairs.org/2022/09/defective-altruism

I think there's a place for private criticism, but it needs to be accompanied by public removal of the offending behavior, and/or apologies. Otherwise, other people don't see that the norms are being reinforced. (And as originally, this applies much more to the more senior / well known EAs.)

JWS 🔸

Thank you for breaking this up into specific sections, I think that this will encourage better discussion on the object-level points, and hopefully keep up community discussion from the initial post. I want to thank the authors for the research and effort that they've put into this series, and I sincerely hope that it has a positive impact on the community.

* * *

From a personal perspective, I feel like these summary points:

The Effective Altruism movement has rapidly grown in size and power, and we have a responsibility to ensure that it lives up to its goals
EA is too homogenous, hierarchical, and intellectually insular, with a hard core of “orthodox” thought and powerful barriers to “deep” critiques
Many beliefs accepted in EA are surprisingly poorly supported, and we ignore entire disciplines with extremely relevant and valuable insights
Some EA beliefs and practices align suspiciously well with the interests of our donors, and some of our practices render us susceptible to conflicts of interest
EA decision-making is highly centralised, opaque, and unaccountable, but there are several evidence-based methods for improving the situation

Can themselves be summarised into 2 different thrusts: ^[1]

Critique 1: The Effective Altruism movement needs to reform its internal systems

From points 1, 2, and 5. Under this view: EA has grown rapidly, however its institutional structure has not adapted optimally, leading to a hierarchical, homogenous, and unaccountable community structure. This means that it does not have the feedback loops necessary to identify cases where there are conflicts of interest, or where the evidence no longer matches the movement's cause prioritisation/financial decisions. In order to change this, the EA movement needs to accept various reforms (which are discussed later in the DEAB piece).

Critique 2: The Effective Altruism movement is wrong

From points 3 and 4. Under this view: Many EA beliefs are not supported by evidence or subject-matter experts. Instead, the movement does little real-world good and its 'revealed preference' is to reputation launder for tech billionaires in the Global North. The movement is rotten to the core, and the best way to improve the world would be to end the enterprise altogether.

(I personally, and presumably most people who have self-selected into reading and posting on the EA Forum, have a lot more sympathy for Critique 1 rather than Critique 2.)

* * *

These are two very different, perhaps almost orthogonal approaches. They remind me of the classic IIDM split between 'improving how institutions make their decisions' and 'improving the decisions that the institutions make'. I think this also links to:

EAs should see EA as a set of intentions and questions (“What does it mean to ‘do the most good’, and how can I do it?”) rather than a set of answers (“AI is the highest-impact cause area, then maybe biorisk.”)

In a simplistic sense, this remains true. But in another sense, it's a bit of EA-Judo.^[2] To the extent the evidence supports having very high credence of AI risk this century (and this isn't by any means unanimous in the community or the forum), or that the world massively underinvests in work to prevent the next global pandemic and mitigate its impacts, then these priorities are what EA is. If were to change what the community thinks about them radically in response to deep critiques, then the resulting movement wouldn't look like EA at all and many, both current supporters and critics, probably wouldn't think EA would be a useful label for the resulting movement.

Perhaps this is related to the overarching theme of being open to "deep critiques". However this summary, and a quick Ctrl+F of the original post, didn't give me a clear definition to me of what a 'deep critique' is^[3] or examples of them which the movement is bad at engaging with. Of the two examples which are referenced, I tend to view Thorstad's blog as following critique 1 - that the EA movement does not take its critics/issues seriously enough and needs to change because of it. However, the forthcoming book (which I am intending to review subject to irl constraints) seems to be more along the lines of critique 2. I think most of the movements most caustic critics, and the ones that probably get most air-time, follow this line of attack.^[4]

From my perspective, a lot of the forum pushback probably comes from the intuition that if EA is open to critique 2 and accepts its conclusions and implications, then it won't be EA anymore. So I think perhaps there needs to be more attention of separating discussions which are closer to critique 1 to critique 2. This isn't not to say that EA shouldn't engage with arguments along the lines of critique 2, but I think it's worth being open about the strengths of those critiques, and the corresponding strength of evidence that would be needed to support them.

^{^}
For clarity, I'm trying to summarise the arguments I think they imply, and not making arguments I personally agree with. I also note that as there a multiple authors from the account they may vary in how much the agree/disagree with the quality of my summarisation.
^{^}
This refers to the tendency to refute any critique of EA by saying, "if you could show this was the right thing to do, then that would be EA." Which may be philosophically consistent but has probably convinced ~0 people in real life.
^{^}
If the authors, or other forum users, could help me out here I would definitely appreciate it :)

^{^}

Some examples:

* https://twitter.com/timnitGebru/status/1614366414671937536

* https://www.youtube.com/watch?v=B_M64BSzcRY

Employees of EA organisations should not be pressured by their superiors to not publish critical work

If it's possible without putting whistleblowers in jeopardy, I would like to know if you have specific examples of this. I've heard a number of worrying claims about stuff 'an EA org' or 'some EA orgs' do, including this, but without knowing the alleged specifics I have no idea how to update on them. Since we've got some tacit norm about not naming the people or even organisations responsible, I suspect that means a lot of problematic behaviours go unaddressed.

AnonymousTurtle

Not exactly an answer, but as an anecdote I know an EA employee who was asked not to publish something 100% supportive of their employer in response to some criticism. We both found it a bit weird, but I assume that's how all organisations work

Yes, that's how all organizations work. Obviously there are cases where employees of an organization should not be publicly commenting to support their organization, because that can be harmful compared to allowing the organization to manage its own reputation. That's not at all the same as suppressing criticism. For example, not responding to trolls is a good thing. I'm all in favor of, say, telling employees not to "defend" EA against claims that it's a secret conspiracy to help rich people. Telling someone not to engage in highlighting dumb bad-faith arguments isn't suppressing their opinions.

I am very concerned that there is implicit pressure not to criticize, but the explicit encouragement by funders and orgs seems to have done a good job pushing back - and the criticism contest was announced well before FTX and the recent attention, and criticisms of EA were common among EA org employees well before any of the Cremer criticism. And I'd note that the highlighted blog is by someone who doesn't identify as EA, but works for GPI.

EAs should read more deep critiques of EA, especially external ones

For instance this blog and this forthcoming book

The blog post and book linked do not seem likely to me to discuss "deep" critiques of EA. In particular, I don't think the problem with the most harmful parts of EA are caused by racism or sexism or insufficient wokeism.

In general, I don't think many EAs, especially very new EAs with little context or knowledge about the community, are capable of recognizing "deep" from "shallow" criticisms, I also expect them to be overly optimistic about the shallow criticisms they preach, and to confuse "deep & unpopular" with 'speculative & wrong'.

Robi Rahman🔸

What do you think are the most harmful parts of EA?

The decisions which caused the FTX catastrophe, the fact that EA is counterfactually responsible for the three primary AGI labs, Anthropic being entirely run by EAs yet still doing net negative work, and the funding of mostly capabilities oriented ML work with vague alignment justifications (and potentially similar dynamics in biotech which are more speculative for me right now), with the creation of ~~GPT and~~^[1] RLHF as particular examples of this.

I recently found out that GPT was not in fact developed for alignment work. I had gotten confused with some rhetoric used by OpenAI and employees during the earlier days which turned out to be entirely independent from modern alignment considerations. ↩︎

Strong disagree for misattributing blame and eliding the question.

To the extent that "EA is counterfactually responsible for the three primary AGI labs," you would need to claim that the ex-ante expected value of specific decisions was negative, and that those decisions were because of EA, not that it went poorly ex-post. Perhaps you can make those arguments, but you aren't.

Ditto for "The decisions which caused the FTX catastrophe" - Whose decisions, where does the blame go, and to what extent are they about EA? SBF's decision to misappropriate funds, or fraudulently misrepresent what he did? CEA not knowing about it? OpenPhil not investigating? Goldman Sachs doing a bad job with due diligence?

I agree with this, except when you tell me I was eliding the question (and, of course, when you tell me I was misattributing blame). I was giving a summary of my position, not an analysis which I think would be deep enough to convince all skeptics.

You say you agree, but I was asking questions about what you were claiming and who you were blaming.

Robi Rahman🔸

EAs are counterfactually responsible for DeepMind?

Off topic, but can you clarify why you think Anthropic does net negative work?

Basically, there are simple arguments around 'they are an AGI capabilities organization, so obviously they're bad', and more complicated arguments around 'but they say they want to do alignment work', and then even more complicated arguments on those arguments going 'well, actually it doesn't seem like their alignment work is all that good actually, and their capabilities work is pushing capabilities, and still makes it difficult for AGI companies to coordinate to not build AGI, so in fact the simple arguments were correct'. Getting more into depth would require a writeup of my current picture of alignment, which I am writing, but which is difficult to convey via a quick comment.

I upvoted and did not disagreevote this, for the record. I'll be interested to see your writeup :)

Do you disagree, assuming my writeup provides little information or context to you?

I don't feel qualified to say. My impression of Anthropic's epistemics is weakly negative (see here), but I haven't read any of their research, but my prior is relatively high AI scepticism. Not because I feel like I understand anything about the field, but because every time I do engage with some small part of the dialogue, it seems totally unconvincing (see same comment), so I have the faint suspicion many of the people worrying about AI safety (sometimes including me) are subject to some mass-Gell-Mann amnesia effect.

Mass Gell-Mann amnesia effect because, say, I may look at others talking about my work or work I know closely, and say "wow! That's wrong", but look at others talking about work I don't know closely and say "wow! That implies DOOM!" (like dreadfully wrong corruptions of the orthogonality thesis), and so decide to work on work that seems relevant to that DOOM?

Yeah, basically that. Even if those same people ultimately find much more convincing (or at least less obviously flawed) arguments, I still worry about the selection effects Nuno mentioned in his thread.