I've been regularly pitched on AI safety roles by people in my network for the last two years.
Do you think it's possible this largely explains why you made the switch and those 20 people who asked you about it hadn't yet? I guess for me this post seemed to be written for a very narrow audience of successful EAs, and largely didn't address the challenges that most EAs have expressed about trying to find a job in AI safety - the biggest of which being that it's extremely competitive.
Do you think it's possible [that your experience being pitched on AI safety roles] largely explains why you made the switch and those 20 people who asked you about it hadn't yet?
Yeah it's reasonable to bring up the possibility, though in my case I don't think it's a compelling narrative. I actually feel pretty embarrassed about this, personally, and I stand by the implication in the main post: Given repeated signals that it would be possible for me to make a switch, I should have been less scared and stuck!!!
Many of the people I’ve spoken to since making my own switch have also been pitched on or referred to AI safety roles, and haven’t switched yet. Others haven’t been pitched, but could likely make a transition with relative ease if they tried hard. Others would have a hard time transitioning. What they all share is the same experience of stuckness, and that's what I'm trying to write about w/ this post.
this post...largely didn't address the challenges that most EAs have expressed about trying to find a job in AI safety - the biggest of which being that it's extremely competitive.
Yes, I think this is right. I'm not trying to address all problems people might face vis-a-vis switching careers to AI safety. And more specifically, I'm not trying to address the particular type of problem where someone is trying hard to switch into an AIS role but is having trouble doing so. Instead I’m trying to address a different problem, where some people aren’t trying hard because they’re stuck with emotional blockers.
Hopefully it'll be clear to people not experiencing the problems I'm addressing that they should ignore my advice, and that it doesn't apply to them!
Of these 5, I would expect 1 and 2 to be "fake objections" much of the time, wheras 3 to 5 can often be meaninfgul concerns. Just because you can't resolve a question doesn't mean its necessarily a bad question or one that should be ignored. I agree if you think deeply and decide that you're just making excuses for yourself and living in fear, then these aren't real questions on a personal level. But they are real for many.
'm in a critical role in global health.[5] If I leave, the work won't get done and people will die. Plus, too many people are already leaving the field.
I won't be able to do anything in AI safety because I don't know anything! I only know stuff about global health.
People in AI safety aren't empirical enough. Specific orgs X, Y, and Z have theories of change that seem vague, and their work might even backfire.
What if my long-term impact would be higher if I stayed at my current org longer, built up more context or seniority, and made a move later?
What if we're just really wrong about AI risk, and alignment is either not an issue at all or unsolvable? Then I should save lives now instead.
Just because you can't resolve a question doesn't mean its necessarily a bad question or one that should be ignored. I agree if you think deeply and decide that you're just making excuses for yourself and living in fear, then these aren't real questions on a personal level. But they are real for many.
Funny, I don't work in GHD but my immediate reaction was the opposite.
1. Reason 1 combines well with reason 5 (or is it not the case that many people are leaving the field? Then ok it would be a fake reason).
2. I think most roles in AI safety do prefer some degree of knowledge, context, or skills that might not transfer from what you've done previously. If many other people have those and you don't, that seems like a genuine reason not to switch.
4. Building up context in a different context makes little sense to me, and building seniority when timelines are this short... and anyway surely experience in AI safety is better? I just feel like this one is motivated reasoning.
Nice post! Although I've encountered a few professionals doing the opposite transition. Many thought they were going to produce a higher impact going into AI Safety, but they discovered their career capital or personal fit was better aligned with the cause they came from, and returned as time went by.
If most people in EA go to work on AIS, the rest of causes will get more neglected and the relevance to tackle them will increase over AIS. The opposite is also true, so a balance should be found.
I would also encourage AIS talent to found new organizations, especially meta / field-building ones to guide talent into the sector. There is an important mismatch between talent and hiring managers expectations, addressing it will be one of the most cost-effective ways to help the field.
tractability is one of the big reasons that drew me into EA, for those of us still trying to understand it as more funding and talent enter AI safety, what does the next talented person moving away from GHD would add?
I feel both terrified about AI risks and confused about how these moves align with my own utilitarian values. Similarly on neglectedness i understand the scale argument but im missing the marginal impact comparison that makes this a stronger case.
here are six quick thoughts, and I want to apologize in advance that these are extremely rushed and unedited -- if anything here seems super off, lmk and I'll either clarify or retract. Also these points are not connected to each other; they're all separate thoughts.
I don't have a numeric model for the marginal impact of working on GHD vs. AIS and I don't think it's that important to build one. But if you wanted to try, you could model the marginal impact of an additional person in GHD by estimating some key proxy outcomes in the EA GHD space that could be caused by a marginal staff member (like...GiveWell's $$ moved and marginal cost-effectiveness, # of high-performing, scalable charities launched by CE/AIM, new money brought in by fundraising orgs) and comparing that to similar proxy outcomes in the AIS space. But on the AIS side it'll be a lot harder to get close to ground truth.
The reality of AI safety right now is that a very small number of people (mainly policymakers and industry leaders) are making decisions that will have colossal impacts on all of humanity. These decision-makers right now are unequipped and unlikely to make decisions that will save the world from catastrophe. A marginal entrant to the AIS space increases the likelihood that we will have better fundamental science of AI alignment; that we will develop realistic policy options that coordinate the companies and countries who together are creating catastrophic risk for the world; that decision-makers will receive the information they need; that decision-makers will be motivated and incentivized to actually take action; and that all of this will happen fast, before it is too late.
Suppose a person could do a very good job working on AIS or GHD. I think we are very far from being in a situation where those two options have equivalent impact. I'd guess on intuition alone that the marginal impact of this person in AIS would be 10x higher (or possibly much higher, but probably not lower).
#3 does not consider the feasibility of switching or any number of other factors that might reasonably affect one's behavior. I'm just saying that I think the abstract impact case for AIS is quite strong and that people who can reasonably switch to work on it should.
I think it'd be worth writing down your reactions to some of the above stuff in a Google Doc and then spending 30 mins poking holes in your own reasoning, and then poking holes in those holes. That might help clarify some of your uncertainties!
I am very confident that there is tractable stuff to do in AI safety. If that's a primary worry holding you back from exploring work opportunities more seriously, I think I'd just encourage you to discard it and instead trust all the people who are loudly saying "we need more people, we are horribly understaffed, there is so much work to do." Tbc, see #4 here, it's not necessarily the case that you'll be a strong personal fit for any particular work opportunity. But I think you should not be abstractly concerned about the tractability of AIS work.
"When I was honest with myself about the real reasons for my hesitation, clarity came quickly and I started making progress toward a career change. "
ducking beautiful!
I really loved how methodological sound, thorough and reflective your process has been!
Without having a real "plan" it sounds like you reverse-engineered a biiig big junk of motivational interviewing, the transtheoretical model and decision making (which is inherently emotional - like it or not).
What you identified as people being stuck can be boiled down to:
Trying to manage an emotion with the decision.
you became "unstuck" once you looked at the emotional aspects of your current situation and future situation. Once you had that emotional clarity you were able to make - as you call it "the judgement call".
A suggestion to add for "How to tackle real reasons" <<Step 1) Write down all the feelings you're trying to avoid by NOT making this decision. 2) Write down all the feelings you're trying to manage WITH this decision >>
then swap out "1) sitting down and writing out your thoughts" with <<3) sit down and write out all your thoughts on the two >> the rest can stay.
there's obviously more nuance that could be provided throughout the post and a sharper focus on the emotional aspects of it, but your post is a wonderful report on how to work through being stuck so it would be inauthentic + wrong tinkering with it without having more of your context (more than "being terrified" for example).
A question I'd have that might further improve the value-density (which is already high) of this post: What made you feel adequately skilled and good enough for AIS? Or were you using courage?
I switched jobs recently, from GiveWell, a global health and development grantmaker, to Coefficient Giving, where I work on AI safety. Since then, at least 20 people[1] working in global health or related areas have reached out for advice, all of them asking a version of the same question: Should I transition to AI safety, and if so, can you help me think through my personal barriers to making a switch?
The first few people I spoke to agreed with me that 1) AI risk is a big, urgent problem, 2) we can do stuff to reduce AI risk, and 3) not enough people are working on AI safety. But they weren't actually…doing much with those beliefs. Instead, they seemed stuck. They weren’t applying to jobs, scheduling info calls with people in their networks, or doing any of the things you’d normally expect from someone working hard to make an important decision. Instead, they were treading water.
After my first few chats, I was somewhat irritated by that phenomenon. But then I remembered that I had been in the same place just a few months ago, which made me feel a bit shamefaced.
See, I wholeheartedly bought the case for AI safety years ago, starting in 2023, and I've been regularly pitched on AI safety roles by people in my network for the last two years.[2] And yet, I only switched jobs a month ago. But why? What blocked me? And how did I fix it?
Well, here's the abridged version of my journey:
I spent a long time inventing reasons to avoid a change that felt scary and emotionally difficult.
A few friends and mentors gave me stern (but warm and loving!) feedback that I should stop being silly, and take my beliefs seriously.
After that, I quickly resolved the small number of my concerns that were actually reasonable.
I switched to work on AI safety.
Because I spent so long waffling, I wondered if other people might be falling victim to the same mental trap of not taking their beliefs seriously and avoiding their emotional blockers.
So, I decided to test out that theory in my remaining conversations: If people were feeling stuck on step one, could I help them get unstuck? As it turned out: Yes!
I mean, not always yes. Pivoting to AI safety wasn’t clearly the right decision for everyone, and it seemed plausible that some people really were having more impact in their current role.[3] But most people I talked to were hesitating due to emotional — rather than practical — concerns, and I was able to help a few of them realize this, work through difficult feelings, and make career decisions more quickly.
This post is for people in the same boat I was in: those who feel an urge to work on AI safety, but haven’t made much progress toward making that happen.
Why you get stuck
I grew up devoutly religious and remained so until I was 25, when I de-converted. I was first exposed to the arguments that ultimately led me to leave my faith years before they convinced me, but I was slow to de-convert[4] because it was extremely emotionally challenging to even consider that option. My entire life (my partner, my social life, my family, my sense of right and wrong) was wrapped up in my religion, and losing it might mean losing my loved ones and sense of self.
So what allowed me to make the leap? Mainly, I stumbled into friend groups and social communities who accepted me for reasons other than my religion. That gave me a security blanket, and made it feel less terrifying to consider radical changes to my beliefs. Once I was no longer terrified, I moved pretty swiftly.
The moral of this story is that for many big decisions, the bigness itself is what prevents us from making progress, and coldly rational arguments don't actually motivate action unless you feel safe. When your animal brain feels like things are too risky, it will protect you by generating reasons to avoid the decision. And these reasons will often be excuses rather than principled, action-guiding beliefs.
I want to be super clear that this sort of unconscious barrier is reasonable. My guess is that having unconscious blockers is an evolutionarily advantageous behavior that, by and large, keeps us from doing chaotic things that we’ll regret later. It's good that our brains work this way!
But sometimes the right thing to do is chaotic, or kinda risky, or at least, unusual — and it's good to notice if you’re holding yourself back because of unconscious, fabricated fears.
How I see this show up
When I talk to people interested in pivoting to AI safety, they often seem similar to my past religious self — they believe AI safety is important and urgent, but they don’t feel ready to act on those beliefs. Instead, they generate a bunch of reasons why they shouldn’t.
Here's a sampling I've heard from people:
I'm in a critical role in global health.[5] If I leave, the work won't get done and people will die. Plus, too many people are already leaving the field.
I won't be able to do anything in AI safety because I don't know anything! I only know stuff about global health.
People in AI safety aren't empirical enough. Specific orgs X, Y, and Z have theories of change that seem vague, and their work might even backfire.
What if my long-term impact would be higher if I stayed at my current org longer, built up more context or seniority, and made a move later?
What if we're just really wrong about AI risk, and alignment is either not an issue at all or unsolvable? Then I should save lives now instead.
These objections are all reasonable on a macrostrategic level — like, if you were a central planner for talent across all philanthropic work, you'd want to consider all of them.
But I think they are largely fake objections, by which I mean they aren’t actually meaningful or motivating for the person who's raising them. Fake objections present as reasonable concerns, but really, they’re unconsciously generated to give people cover to avoid leaving the security of their status quo.[6]
To quickly determine whether an objection is ‘fake’ or not, I started asking people a set of follow-up questions:
Is this concern resolvable? Is there any information you could learn that would satisfy your concern?
This concern seems resolvable! Are you making progress on it? How do you think you could make progress on it?
Suppose this concern was magically solved for you. Would you jump at the chance to take an AI safety role, or would you have outstanding concerns?
In some cases, people have great responses to those questions, like “My concern about whether AI risk is a big deal is totally resolvable; I’ve already written down some thoughts and I have three more calls scheduled to talk through my objections,” or “Huh, yeah, that concern is completely unresolvable, so I should probably accept the uncertainty, knock out my resolvable questions, and then make an informed judgment call.”
But in most cases, I notice that people remain preoccupied by unresolvable questions for a long time, don’t make meaningful progress on their resolvable concerns, and generate endless additional objections after we cover the first ones. They don't actually seem that motivated to work through their concerns; they seem more motivated to create concerns!
What helped me be less fake
When I was distracting myself with fake reasons, the thing that most helped me was close friends giving me gentle but direct feedback. It sounded something like: "You're really thoughtful in general, and I believe that you believe you're thinking carefully about the pros and cons of making a big career change. But I’m surprised that you haven't made much progress on this, despite your conviction that AI safety is the most important problem of our time. I'm worried you might be hiding from your real reasons."
Those conversations helped me uncover the true sources of my delay. After I sorted all my concerns into fake vs. reasonable, I realized that my reticence to switch careers wasn't about potentially draining the global health field of talent, expected value and uncertainty, relative research rigor, population ethics, or any of the philosophical objections that I previously raised.
I was actually blocked because I was in my favorite job ever, with my favorite manager of all time, surrounded by colleagues that I admired, with incredible opportunities for growth. I was worried that I wouldn't be good enough for jobs in AI safety, and the inherent risk of leaving something great for something unproven terrified me. The "impact" argument seemed strong, but it wasn't enough to dislodge me from something that felt comfortable and fun.
When I was honest with myself about the real reasons for my hesitation, clarity came quickly and I started making progress toward a career change.
I hope you'll do a similar self-inspection (faster than I did!) and try to get to the bottom of what's blocking you.
How to tackle real reasons
Once you discard all the fake reasons, you'll almost certainly be left with some real blockers.
But once you're looking at those things clearly, I think you can make fast progress on them through some combination of 1) sitting down and writing out your thoughts, 2) doing desk research, 3) talking to people, 4) deferring to people with more experience than you when you're emotionally blocked.
Here are two examples of things you might discover to be your real reasons, and how I'd go about solving them.
Maybe you're worried that you don't have enough context/knowledge to be useful.
Apply to jobs and trust hiring managers to make good decisions here. They're incentivized to not hire people who don't know enough to be useful, so if you get an offer, you probably shouldn't second-guess it as much as you want to.
Talk to people who switched to AI safety about their experience. My best guess is that you'll often hear something like "I had a basic level of knowledge, I actually needed a lot less than I thought I would, I'm picking up tons of knowledge on the job very fast."
Make a plan to learn the things that you're most worried about not knowing. This isn't hard — start with 80,000 Hours and BlueDot,[7] write down what you’re learning, and discuss it with your friends or friendly strangers on the Internet!
Maybe you don't actually buy the case for AI risk — there seem to be a lot of smart people online saying it's a big deal, but you don't actually feel it in your bones.
Make a fresh Google doc and spend 2 minutes writing down your gut thoughts on why AI risk isn't that important, with zero obligation to be well-calibrated, nuanced, or thoughtful.
Now be nuanced and thoughtful — what would it take to overturn each of your objections? Can you read stuff that would help you? Can you talk to people who have more expert knowledge than you?
These are just slapdash ideas for what you can do to resolve a couple of example concerns. But the same approaches can work for almost any concern — so once you’ve narrowed in on yours, make a plan to resolve them and then execute that plan as quickly as possible.
The final steps
After you resolve all of your specific concerns, I predict that you'll still be dogged by the fear of leaving whatever you're currently doing. That makes sense — for most people, staying the course in their current role will be more secure and stable than whatever pivot option they’re exploring that makes sense.
To be clear, the actual risk is probably a lot lower than you think. Even if your worst fears come true and you aren’t a good fit for roles in AI safety, career changes usually aren’t totally irreversible. If you spend a few months in a role that ends up not being the right fit for you, the odds that you can pivot back to your previous line of work are likely pretty high.
But ultimately, it'll be hard to reduce your feeling of risk to a level that feels emotionally safe. So, I think the final step for most people will look something like asking yourself: Am I willing to accept some risk in order to potentially make a much larger impact with my career?[8]
That’s a question I can’t answer for you, but I want you to know that I and many others will consider you courageous and admirable if you take on risk and cost in order to do good for others. If you're actively considering a big change that feels risky (or deciding whether you should start meaningfully trying to explore a big change), and you think I can help, please tell me![9] I’d also love to hear from you if this post helped you move faster.
I’m really grateful for the editing help that I received from Brit (chiefly), Dave, and Evan. Thank you <3
Most of these people 1) are very influenced by EA ideas like "working on the most neglected, important, and tractable problems in the world is really important", 2) are taking action on those ideas by working on a problem directly or donating a lot of money, and 3) have enough financial cushion that they can consider taking some risk. Those people are also my target audience for this post; I'm definitely not assuming the advice here applies to everyone!
One of those people called me a few days ago and mocked me by playacting a conversation from 1.5 years ago where I gave him a bunch of dumb reasons for not switching to AI safety. It was super embarrassing. I have great friends.
For example, one of my conversations was with a person (we'll call them Bob) on a team that provides general operations support to people focused on a variety of areas, including AI safety, animal welfare, and global health. Bob knew he was having an impact on AI safety by increasing the effectiveness of the AI safety team. He also knew he was really good at his current job. And, Bob wasn't sure if he could find a role focused exclusively on AI safety where his total impact would be higher than in his current role. I asked a bunch of pokey questions and came away pretty confident that Bob was thinking carefully.
Also, I want to be really clear that I'm not trying to say something disrespectful about religion! I'm still uncertain about whether deconverting was actually correct and I still agonize over whether I inappropriately hardened myself against the divine. I'm using this as an example of a situation where my reasoning was blocked by unconscious barriers.
This question will only be meaningful to people who already agree that the impact of working in AI safety may likely exceed the impact of working in other areas. If you don't agree with that, I'd recommend spending time considering the relative impact of various cause areas and ignoring this post’s advice for now.
DM me on Substack or Gmail me at calumbest1. I can’t promise I’ll be able to help you or that I’ll respond, but I read everything people send me and respond where it seems like I can be most useful! You’re most likely to get a quick response from me if you share context about your situation (your goal, what you’ve done so far, and where you’re stuck) and ask questions that you can’t just Google the answers to!
This month effective altruism has gotten lots of attention, including a cover story in The Economist. It feels pretty surreal to be on the receiving end of it. But I feel grateful these ideas have caught on, and even more grateful for the drive, earnestness, and compassion of so many people who took strange but well-argued ideas seriously and then actually acted on them....
We’re the co-founders of Better Futures Guide (BFG), a charity evaluator specifically focused on livelihoods. We were founded last year through Ambitious Impact’s Charity Entrepreneurship Incubation Program. We welcome questions, criticism and opportunities for collaboration, in the comments or at [email protected].
...
TL;DR
* We recently set out RP’s strategic priorities and the three new bets we’re making in 2027.
* In this post, we outline what we need to make them happen.
* We need more support from individual donors, this year in particular.
* We’ve redirected roughly three-quarters of our 2026 unrestrict...
Do you think it's possible this largely explains why you made the switch and those 20 people who asked you about it hadn't yet? I guess for me this post seemed to be written for a very narrow audience of successful EAs, and largely didn't address the challenges that most EAs have expressed about trying to find a job in AI safety - the biggest of which being that it's extremely competitive.
Yeah it's reasonable to bring up the possibility, though in my case I don't think it's a compelling narrative. I actually feel pretty embarrassed about this, personally, and I stand by the implication in the main post: Given repeated signals that it would be possible for me to make a switch, I should have been less scared and stuck!!!
Many of the people I’ve spoken to since making my own switch have also been pitched on or referred to AI safety roles, and haven’t switched yet. Others haven’t been pitched, but could likely make a transition with relative ease if they tried hard. Others would have a hard time transitioning. What they all share is the same experience of stuckness, and that's what I'm trying to write about w/ this post.
Yes, I think this is right. I'm not trying to address all problems people might face vis-a-vis switching careers to AI safety. And more specifically, I'm not trying to address the particular type of problem where someone is trying hard to switch into an AIS role but is having trouble doing so. Instead I’m trying to address a different problem, where some people aren’t trying hard because they’re stuck with emotional blockers.
Hopefully it'll be clear to people not experiencing the problems I'm addressing that they should ignore my advice, and that it doesn't apply to them!