Your work at [large AI safety ecosystem org] is not replaceable

brentonmayer

Your work at [large AI safety ecosystem org] is not replaceable

brentonmayer

6 min readOct 15

Comments

Your work at [large AI safety ecosystem org] is not replaceable — EA Forum

JuanGarcia

It looks like the counterfactual is far from "the manager will have to do it", it's rather "the manager will have to do it for a little while we'll find someone else to do it a little later". Am I missing something obvious?

brentonmayer

1d*

Hmm...

It’s interesting: on the one hand, your hypothetical could be the way it works out, and it doesn't sound that bad. But on the other hand, as I say in the post, I think a sensible way to understand the badness of failing to make the hire is that it’s the difference between a) a 55-person org which is growing and b) a 56-person org which is growing. In your framing, it sounds like the org is only trivially worse off. In my framing, it’s clear that b) is much better off!

I thought about it for a little while. Here are three reasons why I think the costs of failing to make the hire are much higher than your comment makes them sound.

1. Repeating the hiring round fairly soon and making an equivalently strong hire is unusually fortunate

Here’s what might happen instead:

The manager learns they can’t hire for the role they wanted, and scopes out a smaller one. They hand off the procedural parts of the function and keep the rest. What they wanted was for the function to grow into a strong one, with the attention of someone who can oversee it well and make sure it moves to meet the needs of an aggressively growing organisation. It doesn’t get that. And because the smaller role is filled, the pressure to fix it goes away, so the situation tends to continue indefinitely: the function is weaker than it should be, and the manager keeps spending time on it.

Someone else accepts the role. The candidate is, in expectation, weaker, and in these rounds the gap between the top candidate and the next is often surprisingly large (Rob Wiblin worked through this).

The manager just keeps it. No re-run, no re-scope. The function sits at manager-level attention and the manager’s cycle stays blocked. This happens a lot. I’ve had functions that I ended up running with a small fraction of my time, going badly, and that situation continued indefinitely, sometimes in and out of failed hiring rounds.

2. Blocking other functions being scaled

Here’s a cycle which managers in high-growth orgs often spend their attention on: (1) scope out a function (understand what to do and how to do it, see if you can systematise it without hiring, learn what someone overseeing it would look like and what kind of person could do it); (2) hire someone to oversee the function; (3) onboard and train them; (4) use the freed-up time to repeat 1–3 for the next function.

The manager in your hypothetical has failed to get past (2) for this function, so they’ll move on to this cycle for other functions. That's fine, but failing to make the hire means that they'll be slower, in two ways. First, they’ll be spending (say) 10 hours a week overseeing the function for as long as it has no owner. Second, at some point they’ll do a hiring push to find the person, and that push will take something like 70–200 hours. All of that is time they could have spent spinning up the next function. In a fast-growing org, there's always some sense of haste in trying to get these functions going: because as soon as you get someone overseeing it, they're able to make it work well, then grow it at a similar compound growth rate to what the org is (so that it can meet the needs of the org).

3. For whatever time there is no hire, the function is being run poorly by the manager and falling behind org needs

The function which the hiring manager failed to hire for was among the highest priority functions to get more attention to. That's why it was being hired for!

“The manager covers it for a little while” means it runs on (say) ten hours a week of the manager’s time until someone is hired into it. It will fall further behind what the fast-growing org needs for that whole time, and might become a real mess (and/or maybe take more of the hiring manager's attention).

It being increasingly poorly run creates increasing problems for the org, and as it gets more messy it'll take whoever is eventually moved onto it longer to get it on the right track again.

JuanGarcia

22h

Thanks for the very in-depth reply. I find this a lot more convincing than the original post.

Just thought I'd highlight a few empirical questions on which the strength of your argument hinges. After a strong candidate declines an offer:

how often does the role remain persistenly unfilled/under resourced?
how often does it get filled by a similarly capable person within months?
how often does it get filled by a less capable person?
how often does the role end up abandoned?

The exact distribution of expected probability for these outcomes seems to me to determine whether the counterfactual looks more like a temporary slowdown versus a significant reduction in organizational capacity.

brentonmayer

12h

I’m glad you prompted me to write it out, since it seems to land better than the post did. Thanks!

I agree that those seem like relevant empirical questions, and I could imagine them being useful for a candidate to discuss with a potential hiring org.

Though I notice that they fit most naturally with a frame for thinking about organisations which doesn’t feel natural to me. So if a candidate asked me to answer them, I would probably a) try to answer them, but also b) try to share the framing I find most natural. I might say something like:

You could think of an organisation as being made up of a set of roles, each of which is ideally filled by someone strong for that role. Of course, that’s literally true in a sense. But I’m normally thinking of a place like 80k as pursuing a mission which requires an enormous amount of “stuff to be done”, and I think of roles as being contingent fluid constructions which we have to work with. Here's what I mean:
Areas of Responsibility. We create a layer of abstraction over that stuff to be done, called an “Area of Responsibility”. Ideally, an AoR has a single coherent aim, it’s easy to tell where it stops and another responsibility starts, it requires a particular set of skills and context, there’s end-to-end process ownership, responsibility and authority go together, and so on. But often we can’t get all of the criteria that we want met here, and we have to make a judgement call on how to set it up.
Roles. We then create a second layer of abstraction called a “Role”. We have to do this step, because humans are (famously) indivisible, so we need to figure out how best to arrange our people so that the AoRs are overseen (and thus the mission pursued) as best they can be.
Roles are made up of one or more AoRs. Ideally, the AoRs within a role require overlapping skills and context (so that, for example, finance and metrics might go together, or finance and compliance might go together), there’s no split management (i.e. someone reports to the same manager for all of their responsibilities), the responsibilities add up to (usually) at least one full-time equivalent of work (it can be more, because people can hire or oversee teams), and so on.
Critically, each role must be designed so that there is at least one person in the world a) who we think could do a strong job of the role and b) for whom the role is their best option.
Under this view, roles are abstractions whose form is contingent on factors like strategic changes in the world, how pressing each priority within the org is (which changes as people make responsibilities run better, or as strategy changes), the hiring pool we have access to, and so on. For example, in the past month or so, changes in our leadership team have included:
one person taking on the ~entire portfolio of another director (after that director left);
another person looking into spending a full 30% of their capacity on comms (without dropping any of their other portfolio);
another person being unsure whether to spend the majority of their time looking for senior talent to bring into the ecosystem, or on other work.
None of them was previously focused on these things. And this isn’t just senior leadership. In general, the approach which we're likely to take to your role (note - this is the philosophy taken to most, but not all roles at 80k) is something like: a) oversee and improve your responsibilities, b) move them to be ready for greater scale, and c) if something more important comes up which you're i) a great fit for, ii) want to work on, and iii) which otherwise works well given 'structural' considerations, shift to working on that.
So, to your questions: While I’ll try to answer them directly, the above is to flag that I do find it a strained framing. [And then here I actually try to answer the questions, instead of doing an annoying abstract rant.]

Career choice

80,000 Hours

Building the field of AI safety

Counterfactual reasoning

Hiring

Replaceability

Working at EA vs. non-EA orgs

Frontpage

When speaking to candidates who are considering working with us at 80,000 Hours, I often hear people worry about replaceability.

The familiar version of the worry is about whether the organisation will hire someone else in your place. If it will, then the difference you make by taking the job is only the difference between you and them.

There are a couple of well-known answers to that. The person hired in your place probably cares about the world too, and would go on to do something useful with their time instead (Ben Todd wrote about this in 2015). And in a hiring round, the top candidate is often a lot stronger than the next-best one (Rob Wiblin worked this through in 2019).

These days, I think it’s very common for organisations to do bar hiring rather than slot hiring: if we don’t hire you, we won’t hire anyone else.

If someone was considering joining as a podcast host or a writer, that would resolve the replaceability question: it’s clear that their addition to 80k is approximately equal to their podcast episodes or the pieces they write.

But for people in managerial or generalist roles, we often hear a different response to the replaceability worry, which goes something like this:

“80k is already doing pretty well. It’s big, stable, and it has a few committed senior people. If 80k doesn’t hire me, the staff who are already there will keep looking after whatever I would have done. That means I won’t have much impact there and I shouldn’t work there.”

I claim that this line of thinking is mistaken, because it doesn’t reason past the first-order counterfactual.

Of course, it’s literally true that a given responsibility could otherwise be done by someone else.

For example, you could join our People Operations team and run our September payroll. If you didn’t do it, then our Head of People Operations would run payroll. If she couldn’t do it then our Director of Operations would do it (or ask someone on her team to do it). And if she couldn’t do it then our Chief of Staff would do it (or ask someone else to do it).

It’s very unlikely that we don’t do our September payroll, so the first-order analysis shows that it’s entirely replaceable work.

Thinking through the second-order analysis (but skipping the third)^[1], you’ll notice that our Head of People Operations might otherwise have done a deep dive into a tension in a management relationship and substantively improved how the two of them communicate, our Director of Operations might have reviewed the case for a solo Bay Area office and changed her mind on two aspects of it, and our Chief of Staff might otherwise have taken two meetings with speculative candidates to start new programmes.

Somewhere at the bottom of that chain is a piece of work which doesn’t get done: the deep dive, the office review, or the meetings with would-be programme founders.

Diagram: Someone has to run September payroll. If you don’t join, one of three people (Head of People Operations, Director of Operations, Chief of Staff) picks it up, and one of their other pieces of work doesn’t get done. If you join, you run payroll and all three pieces of work get done.

It isn’t trivial to know what 80k will get done because someone is now running our payroll. But it is trivial to reason that something extra will be done. Similarly, if you accept an offer to join, it isn’t trivial for us to reason through what extra will be done because you came. But it is trivial to notice that, in expectation, we’ll be able to achieve more if we’re 56 people than if we’re 55.^[2]

This means that if you are working at 80k, or in fact at any large organisation in the AI safety ecosystem^[3], you’re really likely to be making a contribution which wouldn’t happen without you.

These might seem like straightforward points.

And they’ve been made before and more rigorously than I’m making them here.

But I do still run into the concern that working at a given organisation or in the field generally is replaceable.

So I’ve decided to emphasise what I’m saying here by whining about what it feels like to me when someone tells me that they don’t want to work at 80k because we don’t need them.

What follows is a description of the bottom of this chain at 80k: the things we think are worth doing, and aren’t doing.

Rant

We’ve been working on helping talented people tackle the world’s most pressing problems for 15 years, but the pressing problems still seem erm… Pretty bloody talent constrained.

Like, we are really really trying, and maybe we look like we’ve got our shit together, but in comparison to what our mission and the world are demanding from us, we just suck at what we do. And so we need your help.

I think that most of our existing programmes are markedly below the scale and impact which they could be operating at. I think our research and movement building strategy is further from the bleeding edge than it used to be. I think that we’ve probably lost ground relative to other efforts aimed at getting people into the field in the past couple of years, such that if we repeated the cross-sectional surveys we’d look worse than we previously did.

We totally missed the boat on fellowships. These turned into the ~best scaling movement building innovation of the past 5 years, but we never set one up — even though they’ve been on our radar since I joined in 2017.

We launched our AI career advisor in August 2026, a mere (checks notes) 2.75 years after we first prototyped the idea.

Marketing took us years to get off the ground and we’re still below where we’d like to be by a factor of somewhere between 3 and 30.^[4]

I could go on.

I will go on.

We’ve repeatedly failed to spin up a strong headhunting service for the ecosystem. We had three false starts under different leads, using various approaches, and we’ve only recently begun scaling the approach we’re currently using. We’re well below our potential with the current (more scalable) version of headhunting — by at least a factor of 4.

We have wanted a YouTube channel like AI in Context for years (everyone was nagging us about it); we only managed to launch one last summer. And now that we have one, we wish we were publishing way more on it, being more ambitious with it, and running more channels. Also, we’re still not doing well at shortform video (TikTok etc.) — but surely there’s gotta be a viable strategy here…

The number of people we’ve given one-on-one advice to has flatlined since 2022.

We gave away half a million in talent development grants, then lost the ability to make grants when we spun out of Effective Ventures. We haven’t prioritised rebuilding it, as it would take some work (and probably also because we suck).

We didn’t spin out of Effective Ventures as early as we should have, our recruiting team started in January 2026 (after one false start), and we don’t have a US entity because we don’t have the capacity to set one up.

/rant

Overall, I just think it’s bloody contingent that we ended up being this level of good.

I think we’ve averaged something like 30% growth^[5] a year across 80k.

Said like that, it sounds lovely and smooth, perhaps even overdetermined.

But I don’t think it’s been overdetermined, and I don’t think much of it has been a ‘top down’ decision based on reasoning through when we expect programmes to hit diminishing returns.

Instead, what we’ve been able to do seems extremely contingent on particular staff who managed to make particular contributions, and on particular staff who we failed to hire who didn’t make contributions.

It’s really easy for me to imagine an adjacent version of history where our overall growth rate was more like 50%, or where we shrank. Indeed, in some years, some programmes have more than doubled, in other years programmes have shrunk.

And it seems very possible to me that the returns on resources invested into 80k have increased about as often as they’ve decreased. I think it’s quite possible that our video programme is more cost effective than the rest of 80k: such that 80k saw increasing returns when we added it. And I’m fairly confident that our headhunting team is going through a period of increasing returns.

To summarise: I claim that, properly considered, your work at a given large AI safety ecosystem org is ~not replaceable.

I’ve given a few arguments for that, and I’ve done a lot of whining. I hope that you found at least one of those convincing.

^{^}
I think that it’s mostly fine to skip the third-order analysis, and just stop at the second.
In a stable sector of the economy like healthcare, the number of doctors is set by things like demand and training places. So if you become a doctor, then over the long run someone else probably doesn’t. That’s the third-order analysis, and it’s where replaceability comes back in.
You might want to review this 2014 piece on 80,000 Hours from the economist Rossa O’Keeffe-O’Donovan, on how to reason through how a given entrant affects aggregate supply into a sector over the long run.
But AI safety is not a stable sector of the economy and in the ~~short~~ long run, we’re all dead.
Instead, most of the organisations in AI safety are trying to absorb talent and funding at an extremely high rate, and the ecosystem isn’t in the kind of equilibrium which lends itself to O’Keeffe-O’Donovan’s reasoning above.
^{^}
This assumes the hire goes well. Some hires cost more in management and coordination than they add, at least at first, and that includes some people we’re very glad we hired. But we only make an offer when we expect it to work out, so in expectation the 56th person adds something.
^{^}
See footnote 1.

^{^}

I admit that 3–30 is quite a wide range to be unsure over. Especially for something which we’ve spent over 500 hours analysing. But, as discussed, we suck.

^{^}

By growth, I mean an output metric in quality adjusted careers affected. Like, impact as proxied by the a) number of careers we have affected by b) the amount we’ve affected those careers and c) how talented/motivated the people with those careers are. This is different to impact, but I didn’t want to say ‘increased our output’ because that just sounds like we’ve written more stuff or something. Incidentally, we’ve grown staff at roughly 25% per year.