Comments
I've now spoken to ~1,400 people as an advisor with 80,000 Hours, and if there's a quick thing I think is worth more people doing, it's doing a short reflection exercise about one's current situation.
Below are some (cluster of) questions I often ask in an advising call to facilitate this. I'm often surprised by how much purchase one can get simply from this -- noticing one's own motivations, weighing one's personal needs against a yearning for impact, identifying blind spots in current plans that could be triaged and easily addressed, etc.
A long list of semi-useful questions I often ask in an advising call
- Your context:
- What’s your current job like? (or like, for the roles you’ve had in the last few years…)
- The role
- The tasks and activities
- Does it involve management?
- What skills do you use? Which ones are you learning?
- Is there something in your current job that you want to change, that you don’t like?
- Default plan and tactics
- What is your default plan?
- How soon are you planning to move? How urgently do you need to get a job?
- Have you been applying? Getting interviews, offers? Which roles? Why those roles?
- Have you been networking? How? What is your current network?
- Have you been doing any learning, upskilling? How have you been finding it?
- How much time can you find to do things to make a job change? Have you considered e.g. a sabbatical or going down to a 3/4-day week?
- What are you feeling blocked/bottlenecked by?
- What are your preferences and/or constraints?
- Money
- Location
- What kinds of tasks/skills would you want to use? (writing, speaking, project management, coding, math, your existing skills, etc.)
- What skills do you want to develop?
- Are you interested in leadership, management, or individual contribution?
- Do you want to shoot for impact? How important is it compared to your other preferences?
- How much certainty do you want to have wrt your impact?
- If you could picture your perfect job – the perfect combination of the above – which ones would you relax first in order to consider a role?
- What’s your current job like? (or like, for the roles you’ve had in the last few years…)
- Reflecting more on your values:
- What is your moral circle?
- Do future people matter?
- How do you compare problems?
- Do you buy this x-risk stuff?
- How do you feel about expected impact vs certain impact?
- For any domain of research you're interested in:
- What’s your answer to the Hamming question? Why?
If possible, I'd recommend trying to answer these questions out loud with another person listening (just like in an advising call!); they might be able to notice confusions, tensions, and places worth exploring further. Some follow up prompts that might be applicable to many of the questions above:
- How do you feel about that?
- Why is that? Why do you believe that?
- What would make you change your mind about that?
- What assumptions is that built on? What would change if you changed those assumptions?
- Have you tried to work on that? What have you tried? What went well, what went poorly, and what did you learn?
- Is there anyone you can ask about that? Is there someone you could cold-email about that?
Good luck!
AI safety needs people everywhere but quickly stated, current talent bottlenecks to me look like:
-- Founders
-- Grantmakers
-- (technical) Research leads
-- Policy entrepreneurs and implementors (which includes a lot of technical work)
-- bets in international coordination and/or cooperation
-- All manner of supporting talent -- program leads, ops proper, public outreach, content creators, comms
Most sought-after qualities for talent are:
-- context, mission alignment, domain understanding, sophisticated views on AI strategy and threat modelling etc.
-- "good judgement", "sound epistemics", "reasoning transparency" and other similar ideas/meta-skills from the EA/rationalist cannon
-- a willingness to get shit done/bias for action (rather than be in learning mode, or people who need a lot of management and oversight, or folks with too many preferences/constraints)
-- low ego, similar to above
-- ambitious folks, since they would be really trying to be their own managers, take on bigger projects, grow themselves and their teams etc.
Finally, even having these, it's not enough to just claim to have these; job-seekers mainly trip up in being able to demonstrate and be legible about having them.
I agree with all of this, but I don’t think the gap is talent. It’s in organisations or funders being able to hire, cultivate, or bet on this talent quickly.
Tens of $m’s to a fellowship, talent program, or evaluator seems pretty standard now - where most of that money will go to stipends or compute (read: AI companies). The issue is: what tangible roles aligned to the above are those creating?
Where are the equivalent direct work grants? With $10m I could expand/spin up a team of 30-50 people working directly on advocacy, or communications, or threat modelling and scenario analysis. I’m sure many others could do the same.
I don’t think there much of a talent gap, or a funding gap, but rather a strategic allocation gap.
IANAGM so I can't speak to why they are funding or not funding the specific bets they are/are not funding.
But, speaking abstractly, grantmaking in AI safety is conceptually complicated: Do you take a wide range of bets on unproven theories of change and new grantees, or do you narrowly fund only bets which you have high confidence on? How do you balance between the two?
My intuition is that AI safety puts 1-2 cycles of funding into first-time bets, and quickly moves on to the next set of first-time bets while only continuing to fund orgs and people who used their early money well (and had something to show for it, even negative results). One could argue that funders should be biased towards funding orgs/people for 3 or even 5 cycles before pulling the plug, but this is ultimately a hyperparameter in a quantitative model. It's not obvious to me that the current settings of such hyperparameters are the wrong ones.
With more funding and increased diversity of funders, I absolutely expect more funding philosophies and models in the ecosystem, increasing the kinds of bets being taken.
How would one demonstrate these qualities? It strikes me that building up such a reputation might take an inordinately long time, unless one is already well-known in EA/rat circles.
On "How", my colleague Matt @Matt Beard has some good ideas here: https://80000hours.substack.com/p/how-to-get-into-ai-safety-in-3-months
Yes, it can take time, but one can say more: