This is super relevant for me at this exact moment. Thanks for writing it!
Ask your boss if they think your work might advance the world’s ML capabilities. “yes”: Worry.
Is it possible to get good at ML without advancing ML?
[I'm not an expert in AGI Safety, feel extra free to disagree]
I think using existing AI capabilities is fine, for example:
I think creating new AI capabilities is problematic, for example:
And of course you can learn ML though MLAB (though I hear they're overflowing right now) or other resources, like AI Safety Support maybe.
This post is getting better and better! Thanks so much for it!
This continues Plan B.
If you'd be searching for a job and you'd sit with me as a friend and ask me "hey, you spoke to over 100 developers and saw some of how their job search went, what's the secret element?", then I would answer you, as a friend, that I can guess how well a certain job search will go according to a single KPI: "How many jobs does the person apply to". I'd quickly add "I know it's annoying to apply to lots of jobs, please don't run away", and if our conversation would be one of those conversations-that-go-well, then we would speak about all the bad things about applying to lots of jobs and find solutions to them, or work around them, or something.
If, alternatively, you'd say "oh interesting sounds right" and not give me any of the many pushbacks you're experiencing, then I'd know this is one of those conversations that goes badly, and I'd try to think what I could do differently next time. Maybe I can go meta.
You can check* how much mentorship (or other relevant skill building properties) many work places have, and pick the best (or one of the best). It's hard to know how good this metric is without talking to the hiring manager.
For almost everyone who asks me this, the unusual-property is something that is easier to find out when talking to the hiring manager.
If your response is something like that, please let me know. It's easier for me to write things if I know it's helping someone specific
1. How I see a typical EA software career is (inspired by 80k and I agree), vaguely split into 2:
In the first "half", gain skill.
In the second "half", have direct impact.
The subtext is: Most of your impact will come from the second "half".
2. If you agree, then - in the first half I wouldn't optimize for as-high-impact-as-possible, I'd optimize for building skill, and I'd assume that building skill is the most effective way to have long-term-impact (by reaching the second "half" early). If that seems true for you, see Plan B. Building skill, having fun, and making money - can often go together in software, and I think this is what many people should aim for.
3. When to move to work directly?
It's hard for most of us to estimate our own skill, so what I recommend is "apply to high impact jobs sometimes (every 6/12 months?)". See if you're accepted. If you are, it's time for Plan A. If not, keep having fun with Plan B. The point of applying isn't "getting accepted", it's more about "having the habit of applying sometimes so you don't 'waste' years of having direct impact by building skill that you already have" (this is a common mistake EA devs make, and this habit-of-applying-sometimes is my current best idea for how to solve it).
Part of the reason I think this is a good idea: If you know you're not "missing your chance" of doing something really high impact (because you're applying sometimes) - then I think it's easier to go "all in" to a job that is focused on skill building (and fun and probably money).
TL;DR: Post in social media that you're looking for a job.
What I'd write:
This trick doesn't work for everyone, but it seems super cost effective if it does fit your situation
Thanks for sharing your considerations! I love the approach. Is there a collection/overview of case studies of Software Developers, who had a lot of impact e.g., by community building, transitioning to AI safety, doing earning to give. I imagine that might be quite helpful for software developers, who are thinking about how to optimize their impact
I don't have such a collection myself.
May I ask how this might be helpful for software developers who are thinking about how to optimize their impact?
I also want to guess that most stories are "they applied to an EA aligned job", which is something I'm a pretty strong advocate of
A relatively easy way to get “full points” on these is to pick your favorite top cause, and work at an EA (or an EA vetted) org that is working on that cause.
Most of the article will discuss how to get to that situation.
Other good paths also exist; I want to mention one of them:
It could be a great path (1, 2, 3) but it’s out of scope for this post.
See this post.
Yeah eh please don’t destroy the world or anything thank you!
I recommend you focus on the question "what exactly does this post recommend I personally do" (for example: "apply to X") plus maybe "what is the worst thing about that suggestion for me" (for example: "I don't know if X will accept me")
The rest of the post is split into plans, and the intent is that you try the top plans first, and if they don’t work, go to the next plans.
Also see: Preparing for an AI Safety job at Anthropic or Redwood.
My prior is that most people will have the vast majority of their impact late in their career. If this is true for you: Probably optimize the beginning of your career on building skill, so that you'll reach the "late" part earlier or better equipped to help with whatever comes up.
Consider if this is the right path for you. I think it is for most people.
TL;DR: Consider applying to direct-work jobs every 6-12 months. When you're accepted, it's time to move. Beforehand, build skill, make money, and have fun.
Here are some very common questions:
Here. [Edit: I lost some trust in the default 80k job board following this, but they replied to the post and might have fixed that problem.]
The link above filters for:
It’s a common mistake for EA devs to first filter the list of orgs for a long time, and only then apply, without knowing if those orgs will accept them. I recommend first applying, and then filtering based on where you are accepted.
If there are a few orgs that are so good that you’re not sure which one you’d choose if you were accepted even after thinking about it for 2 minutes (not 2 weeks!) : Start by applying to all of those. If they don’t accept you, go to your next choices.
[This is only true for Plan A. I had people try taking this "Plan A" advice even though this wasn't their stage of the career, which I consider a mistake]
NO!
This is a big problem in the EA dev community: Many devs don’t apply because they are worried about wasting the org’s time, please be part of the solution!
You can, however, be transparent about the situation, for example:
I said both these things myself.
This will often be written in the job ad, and will often be “you can reapply immediately”.
If it’s not written, you can ask (or get me to ask, if you prefer).
For example: “Hey, I’m considering applying, but I’m worried that I’m not ready for the interview. Do you prefer that I take time to study for the interview, or that I apply immediately and study only if I don’t pass?”
I said this myself too.
Probably not. There is a strange problem where EA Devs don’t apply because everyone expects that the next-best person is amazing or something like that. Be part of the solution!
This is a blind spot for many orgs.
Consider telling them. I’m trying to aggregate this information myself, if you’d rather tell me.
A common problem in EA is that each developer has a specialized skillset (like blockchain) and assumes that someone else would be better (like a django+react developer), and this ends up with lots of people not applying at all.
Please be part of the solution!
Who is this plan for: People who can probably get accepted to dozens/hundreds of companies. It's almost always meant for people who already had at least one full time paid "real" job, or people who have something else which makes them very "hirable". Most of the plan is around "how to pick a good company out of all the options". So, if that's you:
I often hear that people want to leave their profession in favor of "the profession most needed". I hope I can get you to think about what will happen if everyone will go to the one profession that is (perceived as) most needed.
I think most people's impact will be determined only a little by "what is the profession needed", it will be determined much more by "is this person really good at their profession (or only kind-of-okay)".
If you agree, I propose a better solution: Each person gets really good at what they have a high personal fit for, and then when an important org needs a really good professional, they have someone to hire. Check sometimes if there's a role open, but otherwise, probably keep going based on other considerations.
If you actually don't mind what profession you'll get into (some of the people I talk to don't) - then yeah, you can prioritize demand. ML Engineering definitely seems in demand.
This question is for: people who are worried they'll learn something that isn't in demand (which I think is mostly a wrong thing to worry about). So:
TL;DR: The skills will probably still be relevant, yes.
I observe that EA tech needs have grown over time, I can imagine them going up even more, and I can’t realistically imagine them going down (assuming EA keeps existing). So my guess is that things that are needed today will also be needed in the future.
Also, my recommendations below will be something like “have an amazing career regardless of EA”, so I don’t expect you to be at risk for “wasting your time learning something useless”. Use your judgment of course, but that is my expectation.
I recently spoke to hiring managers at Anthropic and Redwood Research. They both really want to hire strong developers, even if the developers have zero ML experience and no AI Safety research background.
A significant part of the interviews in both companies are “build a small program in 1 hour”. These are not leetcode questions (for example, they don’t focus on the solution’s complexity), but if you’re good at solving leetcode questions quickly, and writing code quickly in general, then you’ll probably do well in those interviews.
Anthropic said this is similar to what they ask.
Here are some ideas they suggested:
I personally think that practicing writing tiny programs that actually do something in 1 hour would be useful. For example:
They're hiring a Fullstack Developer and, separately, a ML Engineer
TL;DR: When applying, ask “will I have 1 hour every week where I can ask a senior developer questions and get feedback to help me improve?”
This will filter out lots of jobs, such as jobs where you are the only developer.
Also, some places may say they have much more than that: "Just one hour? What? We'll do much more!".
Common mistake: Asking something very subjective
For example: “will I learn a lot?” or “will you help me improve?”.
Almost all hiring managers would reply with “yeah of course!” to those.
Bonus: Try understanding how good your future colleagues are
I don’t think I can explain how to do this, but it’s probably worth trying.
Bad idea: Believe the hiring manager when they say “we only have the best people!” (Fun fact: I used to totally believe these claims myself).
These questions are out of scope for the post, but I’ll at least mention them as having solutions:
Feel free to contact me about these.
Here’s my made-up chart who’s point is “the learning curve is steeper at the beginning, when we’re learning something new”:
When we do something that we already know how to do, we often learn less quickly.
Magic solution: Go do something new.
Yes. But this isn’t the common situation.
“There is a difference between getting 5 years of experience and getting 1 year of experience 5 times”.
For example: If you already know how to build a django backend and you’re building more functions that you mostly already know how to do - then this is not “becoming an expert” and maybe it’s time to switch jobs.
Or, “don’t do something easy” or “don’t do something you already know how to do”, if a main goal of yours is to build skill.
It’s probably better to apply to jobs that are “over your league” or in a domain you don't yet know so that you'll end up doing things you don't know how to do.
Much better: Apply to the places that you most want to work at, and if you’re not accepted, then consider applying to places you have more chances of getting accepted to. You don’t need to apply to a field where 1000 companies are willing to hire you, you only need 1 company, give or take.
It is a neglected area in EA, to find an enjoyable job, even though it has such a large effect on skill down the line.
Consider trying: apply first, without quitting your current job, and see if you’re accepted. Or even cheaper: Send your CV and see if anyone invites you to an interview.
Yeah I endorse income stability. In most situations I recommend first finding a new job and only then quitting.
It is common for me to hear people staying in a job with a bad boss, or not enough learning, or something else that is a really bad fit, but they stay because of one of the reasons above, sometimes the option of leaving is almost unthinkable (“quitting after less than a year??”).
I am not saying all cases are like this. Sometimes problems can be solved, sometimes there are other relevant circumstances, but it seems to be useful to people when I share my prior, so here it is.
These questions are out of scope for the article, but I want to put them here for you to know that these are common questions, often with “textbook” answers, it’s not just you, and I invite you to talk to me about them.
See this post.
If you write a post on one of the topics I mentioned as out of scope, let me know and I’ll link to you
Distributed computing seems to be a skill in high demand among AI safety organizations. Does anyone have recommendations for resources to learn about it? Would it look like using the PyTorch Distributed package or something like a microservices architecture?
To answer your question: My best idea is working on a production distributed product that has bugs around that and debug them, and/or set up a system like that, perhaps with k8s.
This answer is pretty bad, all these ideas are very hard and will take a long time to implement.
My prior is that you might have a bottle neck which is easier to tackle than this one. Wanna share what your goal is, where you stand, and what you're planning to do?
I have a better answer:
Is your bottle neck distributed computing and except for that you can get [the AI Safety job you spoke about] tomorrow? Did you interview with such a company to make sure this is all you have missing?
If not - I'd probably focus on other skills, mainly because distributed computing (1) is not a must, and (2) is pretty hard to learn, compared to other skills, many of which are more important, at least given the ideas I have on how to learn it.
See also the section on preparing for AI Safety jobs if you didn't see it already