The EA Superstructure

Crossposted on Substack.

This recent podcast episode featuring Oliver Habryka and Austin Chen got me thinking about the social and political dimensions of EA. One moment in particular stood out to me.

Habryka: “And so I'm currently expecting that, like, a lot of Anthropic employees will be pretty burned in a year or two. When they are experiencing these dynamics, it becomes very clear that a huge amount of grants are never presented to them because people are worried that they will find them weird. People will see that there's a lot of politics behind the scenes about which grants get put in front of you and stuff like that. People will see that there's a lot of weird reputational management around like, wait, I can't make this grant because another funder would be angry at me, angry at you if you made this grant.”

I think I’m in a poor position to evaluate how true this claim is^[1], and I have some nonspecific worries that what Habryka is doing here may be unproductive, but the pointedness of this claim makes it a good example of something I’d like to call the EA superstructure.

Since most EAs have little love for Marx, I’m free to define the base and superstructure how I want. In my view, the base is composed of the material relationships between actors within EA. Organizations are founded, grants are given, people are hired, etc. The superstructure sits on top of the base, and is composed of the ideas, philosophies, public personas, and vibes. The superstructure, in short, is made up of everything that directs the base beyond economic reality. Habryka’s comment above is clearly part of the superstructure, and a very public attempt to shape it, possibly not for the better.

The superstructure probably isn’t a useful frame for thinking about how to have the most impact, but it can be useful for thinking about the near future of EA, where large amounts of incoming money are likely to change the financial and social landscape. I think the most unsettling worries here have to do with the superstructure being seriously harmed or perverted. In the most disastrous scenario, you might imagine EA being overrun with grifters and promoters, grant applications full of lies, 100 Habrykas, inner rings with inner ring paranoia, factional politics despoiling the commons of public discourse, a den of snakes sunbathing off Telegraph Ave…

I don’t think this extreme scenario will actually come to pass, but briefly thinking about it gives us a sense for how important the superstructure is, and how bad it would be for long run impact if the superstructure deteriorated. A simple response to all of this would be to view the superstructure and the people who influence it as inherently suspect, but this isn’t workable. EA itself, and the principles of impartial cause prioritization, are parts of the superstructure, and plenty of the work done by orgs like Forethought or CEA are attempts to affect the superstructure in positive ways.

Nor do I think we should place much hope in “rat poison”^[2] type interventions to dramatically improve public discourse or honesty, at least in the short run. The track record for these sorts of ideas is poor relative to their aspirations^[3], and while I do think an environment like the EA Forum is an example of better social design, I’m skeptical that there are any large levers to pull here^[4].

It also seems unwise to sit on our hands and pretend that nothing bad will happen with all the change coming. So what can be done? I think the best path forward is to lean into what has made EA great so far. To me, these are things like high openness, a willingness to change course, and being guided by some sort of utilitarian-flavored impact. These values are not a panacea, but I believe that moving away from them would be a bad idea. I worry that in dealing with large inflows of money, people, or publicity, EAs will shift away from these values and bring us closer to the nightmare scenario described above.

To give an extremely small example, my grants database is a case of prioritizing openness over other concerns, and I think NEST is a larger example of championing the sorts of values that we should preserve. There are other orgs and people working in this space, but really not that many. While impact measurement is extremely difficult here, more bright young ambitious people should consider trying to intentionally improve the superstructure. Current community builders should also think of their work more in terms of the superstructure, rather than just motivating and directing people towards impactful careers. If the worriers are right, the need for all of this will soon be clear, but the best time to work on making people better EAs is now.

1. ^{^}

    See this comment for pushback on a related claim about Longview.

2. ^{^}

   Rat poison is poison specifically designed for rationalists, like decision theory or attempting to solve debate.

3. ^{^}

    See this skeptical take on the “Community Notes for everything” idea.

^{^}

For example, it seems like CEA’s Forum team is focused on labor-intensive activities like active moderation, running public events, or generally contributing to the Forum, rather than high-level algorithm or mechanism design changes.

Show all footnotes