Resolving Paradox: Funding Isn’t Bottleneck vs. ~80% high Rejection Rates in AI Safety

Your prior that the problem is not anything other than low pay should be incredibly strong. If, as is prognosticated, a huge sum of money flows into AIS, then Google senior staff engineers and VPs will be in play to lead research projects, and I strongly suspect the prerequisite that the candidate "must be AIS-pilled" will quietly resolve itself. Pricey management consultants at McKinsey and BCG will be retained to speed all this up.

Many people said they wanted to work for METR. I made what I thought was a good offer: take one of the benchmarks we give AIs; if you get a good score then I guarantee that I will fly you out for an interview, even if you have no work history, have no money to pay for the trip, or any other barrier one might have to employment.

Exactly zero people took me up on this.^[1]

How is it possible for there to be sky-high rejection rates yet also zero people sending me applications?

I think the answer is that raw rejection rates aren't a very useful metric. After all, an 80% rejection rate means that the AI safety jobs are 1/10th as selective as Walmart!

I would suggest ignoring raw rejection rates in favor of just looking at the criteria for the jobs you want. Particularly for something like s-risks the criteria are going to be unusual and specific, meaning that even generically qualified people will often have to dedicate substantial time to skilling up, but if you're able to do so, then your odds are pretty good.^[2] 

1. ^{^}

   I wouldn't be surprised to learn that some people tried this, failed, and then were too embarrassed about failing to tell me. But, to the best of my recollection, literally zero people have told me that they even attempted this task.

2. ^{^}

   I say this even with the knowledge that you are 19. I don't want to pretend that the deck isn't stacked against younger people - it totally is - but we employ some 19 year olds, as do other AI safety orgs. If a 19 year old had sent me a good solution to that METR challenge, for example, I would have been happy to hire them.

I've been orienting to the field myself intensely during this year, after following it in the background for years.

Some takes:

• Research that has impact is difficult.
  • AI twitter and AI safety papers on Arxiv surface a high number of papers every week, many of which are using shoddy statistics, bad experiment design, or other basic problems. Many good AIS orgs seem to aim for a high quality standard, and this may well be worth the cost.
  • Research output is also heavy-tailed. If a small fraction of people produce most of the useful work, a high bar can be correct.
• Many organizations are growing as fast as they can. But due to research being difficult, and avoiding organizational drift, keeping quality high, that fast is still not that fast.
• From outside, it's hard to tell someone moving toward good research apart from someone treading water. So the bar is partly about legible evidence, not raw quality. "We can't tell" is a common reason for rejection, distinct from "we can tell and the answer is no."

This is one reason “skill up first” is common advice: it’s a way to  generate signal. Some things that can work: serious attempts at open challenges, replications or extensions of safety papers, benchmark work, public writeups, or programs like MATS, ARENA, SPAR, AI Safety Camp, etc., where the output of the program can itself become a credential. A nice thing about multiple of these routes is that they don’t require permission to start.

In any case, since you've thought of this for 300 hours,  I definitely recommend getting some outside feedback! 80000 hours career advising can be one good option, since they have focused on AI Safety related advising recently.

I think Patient philanthropy and Career capital are valid reasons (not sure about backfire risks).

I'd also highlight that the claim "funding is not the main bottleneck" does not imply that more funding is useless.

And importantly, I think another key part of the 80k advice is to find something you are really good at. So I think the career path priority list should be treated as a starting point that you should first adjust to your own strengths and interests, and then explore through experiments. Start with cheap tests and don't dismiss a great opportunity that is a few items below in the priority list.

If you are considering being a dentist (over alternative EtG paths), then you may already have signals that suggest you'd do well as a dentist. If that's the case, then I think you should also consider Biorisk as a career path. (Starting with reading about it / talking to people who think it's an important problem, if you have not done this already).