Don’t forget why learning is important

This is a timed post. Every 5 minutes while writing this post, I had to stop to do 13 push-ups, and when I could no longer complete my required number, I had to upload it. My friends thought this would be a fun challenge, but it means that it will likely be less polished than some of my other output.

Summary:

Many people moving into AI safety are told that to succeed, they should read a lot and “try to develop takes.” However, it’s important not to forget why knowledge is important: it helps you do better things in the real world. 

Epistemic status: Pretty confident in the main claim (you should focus on learning the most valuable things), but less confident in the specifics of what things are the most valuable. I apologize if my language is too strong at times. 

“I dunno; it seems good to know things?”

The world is filled with ambitious young people, some of whom care about AI risks. Throughout my summer, I’ve gotten to talk to many of them. They visit San Francisco and Berkeley to talk to all the cool AI safety workers, or they go on surprise trips to work on high-impact, time-limited opportunities. Throughout this process, many of them look for jobs, and if they can’t find them, they seek out advice on how to get good enough to land one. 

A common piece of advice for how to get a job in AI safety is “read a lot and try to develop good takes.” This advice is featured very broadly, even for people who never intend to take a technical research role:

• Matt Beard: "You don't have to be confident, but you should have something to say. It's fine to keep your views loosely held and defer to experts well, but you should take ownership over forming your own takes. Hiring managers will also want to hear your thoughts and reasoning."
• 80,000 Hours: 
  • “A rough rule of thumb is to aim to read three or so AI papers a week to get a sense of what’s happening in the field, the terminology people use, and to be able to discriminate between real and fake AI news.”
  • “(No matter how you plan to do AI safety advocacy), it will be important for you to build up your AI safety knowledge.”
• Kairos: “Succeeding in (generalist) roles requires real mission alignment and enough context to spot high-EV opportunities that someone without that background would miss.

As this knowledge propagates, I worry that many people who hear this advice lose track of why it’s important. Many people sharing this advice don’t spend a lot of time explaining why it’s valuable for everyone in AI safety to have their own takes. 

It’s obvious why you would want to know things if you’re trying to become a knowledge expert: knowledge experts are people who know things. However, if you’re trying to become a competent chief of staff, event manager, or field builder, or you want to take one of the many other neglected generalist or org scaling roles, it’s not as obvious why you should read everything. It’s good to have an explanation for why what you’re doing is impactful, but when I ask newcomers why they think it’s important to read, I often get answers like, “I dunno, it seems good to know things?” 

I worry about this. When people don’t have a good picture of why they’re doing something, they might overestimate something’s impact and miss its end goal. If you want to do good things, this is important to avoid.

Knowledge is only as valuable as what it allows you to do.

Knowledge is valuable because it helps you make better decisions. Orgs want people to be knowledgeable because people who know specific things about AI safety can do impactful things, not because storing more facts about AI is intrinsically valuable. You should focus on gaining the knowledge and tools that help you make better decisions. 

To do this, I recommend using a process I call directed reading: identifying the most important questions that will help you make better decisions and doing everything you can to get answers to those. This doesn’t necessarily have to be reading: it can be listening to a podcast, talking to a friend, or chatting with an LLM. But the primary goal is to resolve your most important uncertainties.

Here are some examples of uncertainties you may have:

• If you’re still not sure if AI safety is the most important thing to work on, you can ask yourself, “What is the most important problem in the world to work on?” I don’t think people ask themselves this question enough.
• If you’re new to AI safety and interested in learning all the important “context” that employers keep asking for, you can ask yourself, “Why is context valuable, and how can I most quickly answer the set of important questions that having context answers?”
• If you want to start some impactful projects to build your skills, you can ask yourself, “What projects give me the most valuable skills, and what are the biggest needs in AI safety right now?” 
• If you want to figure out what the biggest needs in AI safety are, you can ask yourself, “What does winning in AI safety look like, and what do we still need to do to make it to the finish line? In other words, what’s my theory of victory?”
• If you hear that we need more generalists in AI safety, and you’re interested in solving this problem, you can ask yourself, “What problems are causing people to believe that we need more generalists, and how can I best address those specific problems?”^[1]
• If you plan to pursue any project, you can ask yourself, “Is this project really on the critical path to making AI go well? Do I have a coherent theory of change? Am I addressing a real problem?”
• If you don’t want to get forever bogged down by infinite questions, you can ask yourself, “When is it safe to defer?”

To answer many of your questions, you’ll probably need to find answers to smaller and more fundamental questions. On its surface, your process might seem to replicate the original “read a lot of things” advice. However, by focusing on what you’re actually hoping to achieve with your knowledge search, you can develop critical skills much faster than you would by passively consuming the next 80,000 Hours podcast episode.^[2]

You can also double-check your learning processes. If the knowledge you’re getting doesn’t help you make better decisions, you should be willing to quickly pivot to trying other things. Learning things does not intrinsically save lives. Misalignment risk isn’t reduced by the number of books you’ve read from cover to cover. Don’t waste time memorizing social passwords and storing fake knowledge.^[3] Do what will help you do good things in the real world.

Other things to keep in mind

While focused learning is important, I don’t think directed reading is the best way to solve all of your problems. 

First, while directed reading can be one of the best ways to resolve your largest uncertainties, you don’t need to spend all of your time skimming posts to find only the answers you need. Reading things you enjoy is pretty awesome, and you shouldn’t force yourself to spend all your time engaging with material that feels like homework. Find ways to integrate your study time into your routine, but don’t forget to do the things that make you happy. 

Second, pure information consumption might not be the best way to gain skills and understanding. Writing can be a valuable tool to help you internalize and understand various ideas, as well as keep track of any uncertainties you might still have.^[4] Additionally, if nobody’s written about the thing you want to say, you should consider polishing your notes and publishing them online. If it helps, you can assemble a group of friends and do a timed writing challenge to publish the scraps you have lying around. 

Finally, you might be completely wrong about what you think you should be studying, or you might be missing some valuable new leads by continuously taking a focused approach to learning. To address this, it can be valuable to introduce some randomness into your media diet: find some time to read things (especially highly recommended things) that you don’t expect to be helpful. However, I suspect that most people aren’t spending enough time trying to learn things directly.

In the immortal words of the great Thomas Henry Huxley (aka “Darwin’s Bulldog”), “The great end of life is not knowledge, but action.”

Time stamps:

• By the end of the first draft, I had done 390 push-ups.
• At publishing, I had done 468 push-ups.

1. ^{^}

   I think I wasted about a week because I didn’t ask myself this question. These kinds of “minimal trust investigations” are valuable for becoming highly talented in a particular domain.

2. ^{^}

   No hate to 80,000 Hours is intended by this; I think these podcasts are great informational resources. However, if you have the opportunity to sit and read, I think you should skim the transcript and find the parts that answer your questions. 

3. ^{^}

   This might make it easier to get a job, but if you try to study these things specifically, you’re missing the thing that actually helps you become impactful

^{^}

I also like Andy Masley’s blog post about this

Show all footnotes