This is a special post for quick takes by cb. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Here's some quick takes on what you can do if you want to contribute to AI safety or governance (they may generalise, but no guarantees). Paraphrased from a longer talk I gave, transcripthere.
First, there’s still tons of alpha left in having good takes.
(Matt Reardon originally said this to me and I was like, “what, no way”, but now I think he was right and this is still true – thanks Matt!)
You might be surprised, because there’s many people doing AI safety and governance work, but I think there’s still plenty of demand for good takes, and you can distinguish yourself professionally by being a reliable source of them.
But how do you have good takes?
I think the thing you do to form good takes, oversimplifying only slightly, is you read Learning by Writing and you go “yes, that’s how I should orient to the reading and writing that I do,” and then you do that a bunch of times with your reading and writing on AI safety and governance work, and then you share your writing somewhere and have lots of conversations with people about it and change your mind and learn more, and that’s how you have good takes.
What to read?
Start with the basics (e.g. BlueDot’s courses, other reading lists) then work from there on what’s interesting x important
Write in public
Usually, if you haven’t got evidence of your takes being excellent, it’s not that useful to just generally voice your takes. I think having takes and backing them up with some evidence, or saying things like “I read this thing, here’s my summary, here’s what I think” is useful. But it’s kind of hard to get readers to care if you’re just like “I’m some guy, here are my takes.”
Some especially useful kinds of writing
In order to get people to care about your takes, you could do useful kinds of writing first, like:
Explaining important concepts
E.g., evals awareness, non-LLM architectures (should I care? why?) , AI control, best arguments for/against short timelines, continual learning shenanigans
Collecting evidence on particular topics
E.g., empirical evidence of misalignment, AI incidents in the wild
Summarizing and giving reactions to important resources that many people won’t have time to read
For example, if someone wrote a blog post on “I read Anthropic’s sabotage report, and here’s what I think about it,” I would probably read that blog post, and might find it useful.
Writing vignettes, like AI 2027, about your mainline predictions for how AI development goes.
Ideas for technical AI safety
Reproduce papers
Port evals to Inspect
Do the same kinds of quick and shallow exploration you’re probably already doing, but write about it—put your code on the internet and write a couple paragraphs about your takeaways, and then someone might actually read it!
Some quickly-generated, not-at-all-prioritised ideas for topics
Stated vs revealed preferences in LLMs
How sensitive to prompting is Anthropic’s blackmail results?
Testing eval awareness on different models/with different evaluations
Can you extend persona vectors to make LLMs better at certain tasks? (Is there a persona vector for “careful conceptual reasoning”?)
Is unsupervised elicitation a good way to elicit hidden/password-locked/sandbagged capabilities?
You can also generate these topics yourself by asking, “What am I interested in?”
Nobody’s on the ball
I think there are many topics in AI safety and governance where nobody’s on the ball at all.
And on the one hand, this kind of sucks: nobody’s on the ball, and it’s maybe a really big deal, and no one is handling it, and we’re not on track to make it go well.
But on the other hand, at least selfishly, for your personal career—yay, nobody’s on the ball! You could just be on the ball yourself: there’s not that much competition.
So if you spend some time thinking about AI safety and governance, you could probably pretty easily become an expert in something pretty fast, and end up having pretty good takes, and therefore just help a bunch.
I am too young and stupid to be giving career advice, but in the spirit of career conversations week, I figured I'd pass on advice I've received which I ignored at the time, and now think was good advice: you might be underrating the value of good management!
I think lots of young EAish people underrate the importance of good management/learning opportunities, and overrate direct impact. In fact, I claim that if you're looking for your first/second job, you should consider optimising for having a great manager, rather than for direct impact.
Why?
Having a great manager dramatically increases your rate of learning, assuming you're in a job with scope for taking on new responsibilities or picking up new skills (which covers most jobs).
It also makes working much more fun!
Mostly, you just don't know what you don't know. It's been very revealing to me how much I've learnt in the last year, I think it's increased my expected impact, and I wouldn't have predicted this beforehand.
In particular, if you're just leaving university, you probably haven't really had a manager-type person before, and you've only experienced a narrow slice of all possible work tasks. So you're probably underrating both how useful a very good manager can be, and how much you could learn.
How can you tell if someone will be a great manager?
This part seems harder. I've thought about it a bit, but hopefully other people have better ideas.
Ask the org who would manage you and request a conversation with them. Ask about their management style: how do they approach management? How often will you meet, and for how long? Do they plan to give minimal oversight and just check you're on track, or will they be more actively involved? (For new grads, active management is usually better.) You might also want to ask for examples of people they've managed and how those people grew.
Once you're partway through the application process or have an offer, reach out to current employees for casual conversations about their experiences with management at the org.
You could ask how the organization handles performance reviews and promotions. This is probably an okay-not-great proxy, since smaller, fast-growing orgs might have informal processes but still excellent management, but I thin k it would give you some signal on how much they think about management/personal development.
(This maybe only really works if you are socially very confident or know lots of EA-ish people, sorry about that) You could consider asking a bunch of your friends and acquaintances about managers they've had that they thought were very good, and then trying to work with those people.
Some random heuristics: All else equal, high turnover rate without seemingly big jumps in career progression seems bad. Orgs that regularly hire and retain/promote early career people are probably pretty good at management; same for orgs whose alumni go on to do cool stuff.
Agreed, but I'd be careful not to confuse good mentorship with good management. These usually go hand-in-hand. But sometimes a manager is good because they sacrifice some of your career growth for the sake of the company.
I like the archetypes of ruthless versus empathetic managers described here. It's an arbitrary division and many managers do fulfill both archetypes. But I think it also captures an important dynamic, where managers have to tradeoff between their own career, your career, and the organization as a whole. Mentorship and career development falls into that
Edit: Another distinction I'd add is good manager versus good management. Sometimes it's the organizational structure that determines whether you'll get good training. In my experience, larger and stable organizations are better at mentorship for a ton of reasons, such as being able to make multi-year investments in training programs. A scrappy startup, on the other hand, may be a few weeks away from shutting down.
I definitely feel a few of my past managers would have been much better at mentorship if other aspects of the situation were different (more capacity, less short-term deadlines, better higher-up managers, etc.).
Here's some quick takes on what you can do if you want to contribute to AI safety or governance (they may generalise, but no guarantees). Paraphrased from a longer talk I gave, transcript here.
(All views here my own.)
I am too young and stupid to be giving career advice, but in the spirit of career conversations week, I figured I'd pass on advice I've received which I ignored at the time, and now think was good advice: you might be underrating the value of good management!
I think lots of young EAish people underrate the importance of good management/learning opportunities, and overrate direct impact. In fact, I claim that if you're looking for your first/second job, you should consider optimising for having a great manager, rather than for direct impact.
Why?
How can you tell if someone will be a great manager?
(My manager did not make me post this)
Agreed, but I'd be careful not to confuse good mentorship with good management. These usually go hand-in-hand. But sometimes a manager is good because they sacrifice some of your career growth for the sake of the company.
I like the archetypes of ruthless versus empathetic managers described here. It's an arbitrary division and many managers do fulfill both archetypes. But I think it also captures an important dynamic, where managers have to tradeoff between their own career, your career, and the organization as a whole. Mentorship and career development falls into that
Edit: Another distinction I'd add is good manager versus good management. Sometimes it's the organizational structure that determines whether you'll get good training. In my experience, larger and stable organizations are better at mentorship for a ton of reasons, such as being able to make multi-year investments in training programs. A scrappy startup, on the other hand, may be a few weeks away from shutting down.
I definitely feel a few of my past managers would have been much better at mentorship if other aspects of the situation were different (more capacity, less short-term deadlines, better higher-up managers, etc.).