No worries Ollie, thanks for the feedback :)

As I said, those bullet points were a memory of a draft so I don't have the hard data to share on hand. But when dealing with social movements it's always going to be somewhat vibesy - data will necessarily be observational and we can't travel back in time and run RCTs on whether SBF commits fraud or not. And the case studies do show that declines can go on for a very long time post major crisis. It's rare for movements to disappear overnight (The Levellers come closest of all the cases I found to that)

Fwiw I think that the general evidence does point to "EA is in decline" broadly understood, and that should be considered the null hypothesis at this point. I'd feel pretty gaslit if someone said EA was going swimmingly and unaffected by the tribulations of the last couple of years, perhaps less so if they think there's been a bounce back after an initial decline but, you know, I'd want to see the data for that.

But as I said, it's really not the (main) point of the post! I'd love to add my points to a post where someone did try and do a deep dive into that question.

Hey Ollie, thanks for your feedback! It helped me understand some of the downvotes the post was getting which I was a bit confused by. I think you and perhaps others are interpreting the post as "Here are some case studies that show EA is in decline", but that's not what I was trying to write, it was more "EA is in decline, what historical cases can inform us about this?" I'm not really arguing for "Is EA in decline?" in the post, in fact I'm just assuming it and punting the empirical evidence for another time, since I was interested in bringing out the historical cases rather than EAs current state. So the tweets are meant to be indicative of mood/sentiment but not load bearing proof. I do see that the rhetorical flourish in the intro might have given a misleading impression, so I will edit that to make the point of the piece more clear.

As for why, I mean, it does just seem fairly obvious to me, but social movements have fuzzy boundaries and decline doesn't have to be consistent. Nevertheless answering this question was a post I was planning on write and the evidence seemed fairly damning to me - for instance:

• Spikes in visiting the "Effective Altruism" Wikipedia page seem to mainly be in response to negative media coverage, such as the OpenAI board fallout or a Guardian Article about Wytham Abbey. Other Metrics like Forum engagement that were growing preFTX clearly spike and decline after the scandal period.
• Other organisations similar to EA considering rebounding away. Apparently CE/AIM was considering this, and Rutger Bregman seems to be trying really hard to keep Moral Ambition free of EAs reputational orbit, which I think he'd be doing much less of if EAs prospects were more positive.
• Previous community members leaving the Forum or EA in general, sometimes turning quite hostile to it. Some examples here are Habryka, Nuño Sempere, Nathan Young, Elizabeth from the more 'rationalist' side. I've noticed various people who were long term EAs, like Akash Wasil and John Halstead have deactivated their accounts. A lot of more left-wing EAs like Bob Jacobs seem to have started to move away too, or like Habiba moderate their stanc and relationship to EA.
• This goes double so for leadership. I think loads kf the community felt a leadership vacuum post FTX. Dustin has deactivated his account. Holden has left OP, gone quiet, and might not longer consider himself EA? Every so often Ben Todd tweets something which I can only interpret as "testing the waters before jumping ship". I don't think leadership of a thriving, growing movement acts this.

• If you search "Effective Altruism" on almost any major social media site (X, Bluesky, Reddit, etc) I suspect the general sentiment toward EA will be strongly negative and probably worse than it was preFTX and staying that way. There might be some counter evidence. Some metrics might have improved, and I know some surveys are showing mixed or positive things. I think Habryka's point that reputation is evaluated lazily rings true to me even if I disagree on specifics.

  ---

  But again, the above is my memory of a draft, and I'm not sure I'll ever finish that post. I think hard data on a well formed version of the question would be good, but once again it's not the question I was trying to get at with this post.

Some Positions In EA Leadership Should Be Elected

If we take this literally, then 'some' means 'any at all', and I think that the amount of democratic input is >0 so this should be 100% yes.

Having said that, I think the bar is kind-of acting as a 'how much do you support democratising EA', and in that sense while I do support Cremer-style reforms, I think they're best introduced at a small scale to get a track record.

I wish this post - and others like it - had more specific details when it comes to these kind of criticisms, and had a more specific statement of what they are really taking issue with, because otherwise it sort of comes across as "I wish EA paid more attention to my object-level concerns" which approximately ~everyone believes.

If the post it's just meant to represent your opinions thats perfectly fine, but I don't really think it changed my mind on its own merits. I also just don't like withholding private evidence, I know there are often good reasons for it, but it means I just can't give much credence to it tbqh. I know it's a quick take but still, it left me lacking in actually evaluating.

I general this discussion reminds me a bit of my response to another criticism of EA/Actions by the EA Community, and I think what you view as 'sub-optimal' actions are instead best explained by people having:

1. bounded rationality and resources (e.g. You can't evaluate every cause area, and your skills might not be as useful across all possible jobs)
2. different values/moral ends (Some people may not be consequentialists at all, or prioritise existing vs possible people difference, or happiness vs suffering)
3. different values in terms of process (e.g. Some people are happy making big bets on non-robust evidence, others much less so)

And that, with these constraints accepted, many people are not actually acting sub-optimally given their information and values (they may later with hindsight regret their actions or admit they were wrong, of course)

Some specific counterexamples:

• Being insufficiently open-minded about which areas and interventions might warrant resources/attention, and unwarranted deference to EA canon, 80K, Open Phil, EA/rationality thought leaders, etc. - I think this might be because of resource constraints, or because people disagree about what interventions are 'under invested' vs 'correctly not much invested in'. I feel like you just disagree with 80K & OpenPhil on something big and you know, I'd have liked you to say what that is and why you disagree with them instead of dancing around it.
• Attachment to feeling certainty, comfort and assurance about the ethical and epistemic justification of our past actions, thinking, and deference - What decisions do you mean here? And what do you even mean by 'our'? Surely the decisions you are thinking of apply to a subset of EAs, not all of EA? Like this is probably the point that most needed something specific.

Trying to read between the lines - I think you think the AGI is going to be a super big deal soon, and that its political consequences might be the most important and consequently political actions might be the most important ones to take? But OpenPhil and 80K haven't been on the ball with political stuff and now you're disillusioned? And people in other cause areas that aren't AI policy should presumably stop donating there and working their and pivot? I don't know, it doesn't feel accurate to me,^[1] but like I can't get a more accurate picture because you just didn't provide any specifics for me to tune my mental model on 🤷‍♂️

Even having said all of this, I do think working on mitigating concentration-of-power-risks is a really promising direction for impact and wish you the best in pursuing it :)

1. ^{^}

   As in, i don't think you'd endorse this as a fair or accurate description of your views

Note - this was written kinda quickly, so might be a bit less tactful than I would write if I had more time.

Making a quick reply here after binge listening to three Epoch-related podcasts in the last week, and I basically think my original perspective was vindicated. It was kinda interesting to see which points were repeated or phrased a different way - would recommend if your interested in the topic.

• The initial podcast with Jaime, Ege, and Tamay. This clearly positions the Epoch brain trust as between traditional academia and the AI Safety community (AISC). tl;dr - academia has good models but doesn't take ai seriously, and AISC the opposite (from Epoch's PoV)
• The 'debate' between Matthew and Ege. This should have clued people in, because while full of good content, by the last hour/hour and half it almost seemed to turn into 'openly mocking and laughing' at AISC, or at least the traditional arguments. I also don't buy those arguments, but I feel like the reaction Matthew/Ege have shows that they just don't buy the root AISC claims.
• The recent podcast Dwarkesh with Ege & Tamay. This is the best of the 3, but probably also best listened too after the first too, since Dwarkesh actually pushes back on quite a few claims, which means Ege & Tamay flush out their views more - personal highlight was what the reference class for AI Takeover actually means.

Basically, the Mechanize cofounders don't agree at all with 'AI Safety Classic', I am very confident that they don't buy the arguments at all, that they don't identify with the community, and somewhat confident that they don't respect the community or its intellectual output that much. 

Given that their views are: a) AI will be a big deal soon (~a few decades), b) returns to AI will be very large, c) Alignment concerns/AI risks are overrated, and d) Other people/institutions aren't on the ball, then starting an AI Start-up seems to make sense.

What is interesting to note, and one I might look into in the future, is just how much these differences in expectation of AI depend on differences in worldview, rather than differences in technical understanding of ML or understanding of how the systems work on a technical level.

So why are people upset?

• Maybe they thought the Epoch people were more part of the AISC than they actually were? Seems like the fault of the people believe this, not Epoch or the Mechanize founders.
• Maybe people are upset that Epoch was funded by OpenPhil, and this seems to have lead to 'AI acceleration'? I think that's plausible, but Epoch has still produced high-quality reports and information, which OP presumably wanted them to do. But I don't think equating EA == OP, or anyone funded by OP, is a useful concept to me.
• Maybe people are upset at any progress in AI capabilities. But that assumes that Mechanize will be successful in its aims, not guaranteed. It also seems to reify the concept of 'capabilities' as one big thing which i don't think makes sense. Making a better Stockfish, or a better AI for FromSoft bosses does not increase x-risk, for instance.
• Maybe people think that the AI Safety Classic arguments are just correct and therefore people taking actions other than it. But then many actions seem bad by this criteria all the time, so odd this would provoke such a reaction. I also don't think EA should hang its hat on 'AI Safety Classic' arguments being correct anyway.

Probably some mix of it. I personally remain not that upset because a) I didn't really class Epoch as 'part of the community', b) I'm not really sure I'm 'part of the community' either and c) my views are at least somewhat similar to the Epoch set above, though maybe not as far in their direction, so I'm not as concerned object-level either.

I'm not sure I feel as concerned about this as others. tl;dr - They have different beliefs from Safety-concerned EAs, and their actions are a reflection of those beliefs.

It seems broadly bad that the alumni from a safety-focused AI org

Was Epoch ever a 'safety-focused' org? I thought they were trying to understand what's happening with AI, not taking a position on Safety per se.

 ...have left to form a company which accelerates AI timelines

I think Matthew and Tamay think this is positive, since they think AI is positive. As they say, they think explosive growth can be translated into abundance. They don't think that the case for AI risk is strong, or significant, especially given the opportunity cost they see from leaving abundance on the table.

Also important to note is what Epoch boss Jaime says in this very comment thread.

As I learned more and the situation unfolded I have become more skeptical of AI Risk.

The same thing seems to be happening with me, for what it's worth.

People seem to think that there is an 'EA Orthodoxy' on this stuff, but there either isn't as much as people think, or people who disagree with it are no longer EAs. I really don't think it makes sense to clamp down on 'doing anything to progress AI' as being a hill for EA to die on.

Note: I'm writing this for the audience as much as a direct response

The use of Evolution to justify this metaphor is not really justified. I think Quintin Pope's Evolution provides no evidence for the sharp left turn (which won a prize in an OpenPhil Worldview contest) convincingly argues against it. Zvi wrote a response from the "LW Orthodox" camp that wasn't convincing and Quintin responds against it here.

On "Inner vs Outer" framings for misalignment is also kinda confusing and not that easy to understand when put under scrutiny. Alex Turner points this out here, and even BlueDot have a whole "Criticisms of the inner/outer alignment breakdown" in their intro which to me gives the game away by saying "they're useful because people in the field use them", not because their useful as a concept itself.

Finally, a lot of these concerns revolve around the idea of their being set, fixed, 'internal goals' that these models have, and represent internally, but are themselves immune from change, or can hide from humans, etc. This kind of strong 'Goal Realism' is a key part of the case for 'Deception' style arguments, whereas I think Belrose & Pope show an alternative way to view how AIs work is 'Goal Reductionism', in which framing the issues imagined don't seem certain any more, as AIs are better understood as having 'contextually-activated heuristics' rather than Terminal Goals. For more along these lines, you can read up on Shard Theory.

I've become a lot more convinced about these criticisms of "Alignment Classic" by diving into them. Of course, people don't have to agree with me (or the authors), but I'd highly encourage EAs reading the comments on this post to realise Alignment Orthodoxy is not uncontested, and is not settled, and if you see people making strong cases based on arguments and analogies that seem not solid to you, you're probably right, and you should look to decide for yourself rather than accepting that the truth has already been found on these issues.^[1]

1. ^{^}

   And this goes for my comments too

I'm glad someone wrote this up, but I actually don't see much evaluation here from you, apart from "it's too early to say", but then Zhou Enlai pointed out that you could say that about the French Revolution,^[1] and I think we can probably say some things. I generally have you mapped to the "right-wing Rationalist" subgroup Arjun,^[2] so it'd be actually interested to get your opinion instead of trying to read between the lines on what you may or may not believe. I think there was a pretty strong swing in Silicon Valley / Tech Twitter & TPOT / Broader Rationalism towards Trump, and I think this isn't turning out well, so I'd actually be interested to see people saying what they actually think - be that "I made a huge mistake", "It was a bad gamble but Harris would've been worse" or even "This is exactly what I want"

1. ^{^}

   I know it's not apocryphal but it's a good quote

2. ^{^}

   Let me know if this is wrong and/or you don't identify this way

Hey Cullen, thanks for responding! So I think there are object-level and meta-level thoughts here, and I was just using Jeremy as a stand-in for the polarisation of Open Source vs AI Safety more generally.

Object Level - I don't want to spend too long here as it's not the direct focus of Richard's OP. Some points:

• On 'elite panic' and 'counter-enlightenment', he's not directly comparing FAIR to it I think. He's saying that previous attempts to avoid democratisation of power in the Enlightenment tradition have had these flaws. I do agree that it is escalatory though.
• I think, from Jeremy's PoV, that centralization of power is the actual ballgame and what Frontier AI Regulation should be about. So one mention on page 31 probably isn't good enough for him. That's a fine reaction to me, just as it's fine for you and Marcus to disagree on the relative costs/benefits and write the FAIR paper the way you did.
• On the actual points though, I actually went back and skim-listened to the the webinar on the paper in July 2023, which Jeremy (and you!) participated in, and man I am so much more receptive and sympathetic to his position now than I was back then, and I don't really find Marcus and you to be that convincing in rebuttal, but as I say I only did a quick skim listen so I hold that opinion very lightly.

Meta Level - 

• On the 'escalation' in the blog post, maybe his mind has hardened over the year? There's probably a difference between ~July23-Jeremy and ~Nov23Jeremy, which he may view as an escalation from the AI Safety Side to double down on these kind of legislative proposals? While it's before SB1047, I see Wiener had introduced an earlier intent bill in September 2023.
• I agree that "people are mad at us, we're doing something wrong" isn't a guaranteed logic proof, but as you say it's a good prompt to think "should i have done something different?", and (not saying you're doing this) I think the absolutely disaster zone that was the sB1047 debate and discourse can't be fully attributed to e/acc or a16z or something. I think the backlash I've seen to the AI Safety/x-risk/EA memeplex over the last few years should prompt anyone in these communities, especially those trying to influence policy of the world's most powerful state, to really consider Cromwell's rule.
• On this "you will just in fact have pro-OS people mad at you, no matter how nicely your white papers are written." I think there's some sense in which it's true, but I think that there's a lot of contigency about just how mad people get, how mad they get, and whether other allies could have been made on the way. I think one of the reasons they got so bad is because previous work on AI Safety has understimated the socio-political sides of Alignment and Regulation.^[1]

1. ^{^}

   Again, not saying that this is referring to you in particular

I responded well to Richard's call for More Co-operative AI Safety Strategies, and I like the call toward more sociopolitical thinking, since the Alignment problem really is a sociological one at heart (always has been). Things which help the community think along these lines are good imo, and I hope to share some of my own writing on this topic in the future.

Whether or not I agree with Richard's personal politics or not is kinda beside the point to this as a message. Richard's allowed to have his own views on things and other people are allowed to criticse this (I think David Mathers' comment is directionally where I lean too). I will say that not appreciating arguments from open-source advocates, who are very concerned about the concentration of power from powerful AI, has lead to a completely unnecessary polarisation against the AI Safety community from it. I think, while some tensions do exist, it wasn't inevitable that it'd get as bad as it is now, and in the end it was a particularly self-defeating one. Again, by doing the kind of thinking Richard is advocating for (you don't have to co-sign with his solutions, he's even calling for criticism in the post!), we can hopefully avoid these failures in the future.

On the bounties, the one that really interests me is the OpenAI board one. I feel like I've been living in a bizarro-world with EAs/AI Safety People ever since it happened because it seemed such a collosal failure, either of legitimacy or strategy (most likely both), and it's a key example of the "un-cooperative strategy" that Richard is concerned about imo. The combination of extreme action and ~0 justification either externally or internally remains completely bemusing to me and was big wake-up call for my own perception of 'AI Safety' as a brand. I don't think people can underestimate the second-impact effect this bad on both 'AI Safety' and EA, coming about a year after FTX.