Roman Ross's Quick takes

Aimed at an audience newer to EA/AI safety

It seems like many people have been confused lately about why 1) AI safety orgs are in desperate need of good hires and 2) org hiring rates are very low. My understanding is that there are many reasons for this, but a 2021 post states it pretty well:

"A lot of EA content is subtle and detailed, and easy to accidentally compress into something misleading. For example, 80k might write a nuanced article saying 'You should focus on talent gaps, not funding gaps', but this gets translated into 'EA is talent constrained', and then people repeat that phrase without linking to the article, and then many people form an inaccurate belief that EA needs 'pretty talented people', rather than 'EA needs very specific talents that are missing.'"

The best career advice seems to be to focus on developing a rare and valuable skill that orgs currently need. In AI safety, this skill is a combination of specific context, agency, mission-alignment, and strategic vision. You can develop it by reading a lot, writing out your thoughts, talking to people at events, and completing real projects.

It's hard to do really impactful work. You need a plausible explanation for why your efforts are the best way your salary can be spent. This takes time, effort, and careful strategizing. Most of the world has high schools, colleges, and internships to train people for jobs, but a similar system is only recently becoming legible for AI safety and EA. The rare skills are rare for a reason. Still, I think the broader EA community could do a better job of communicating what it means for these "talent-gaps" to exist.

Quick math: In terms of total expected suffering averted, inviting a group of friends to a barbecue where the main meat is beef is probably more impactful than eating a vegan meal by yourself.

According to Faunalytics' Animal Product Impact Scales, a serving of chicken requires approximately 10 times as many days of animal life to produce as a serving of beef (not adjusted for quality of life or moral weights). Additionally, most seafood is much, much worse than this. If this meal prevents your friends from going and eating those other foods, tricking them into eating beef will likely do more good than eating a vegan meal by yourself. 

Still, this may not be as impactful as saving your money and donating it directly to impactful charities. Saving a single dollar on food and donating it to the SWP will probably do more to reduce suffering than hosting a barbecue. 

Also, you might expose yourself to various forms of value drift by hosting a barbecue, but I could see this happening in a net positive (hosting more barbecues in the future) or a net negative (giving up on veganism entirely/permanently failing to convince any of your friends to give up meat) direction.

At my particular place, I think that improving AI safety community epistemics (writing thorough and careful arguments against people both in the community and outside of it who make silly mistakes) is, by most reasonable estimates, probably less impactful than helping to run MAIA, which is what I currently spend most of my time doing.

I don’t think I’m the only person in this situation, which is quite scary to me, as I think there are a lot of things are epistemically iffy in the AI safety community. For example:

• A lot of the most viral pro-AIS people (especially the ones that are relatively new to the field) seem to not be reasoning especially carefully
• The field’s hiring process is quite insular
• Not enough time is spent engaging with a lot of good faith disagreements (imo)
• Fieldbuilding strategy doesn’t optimize for careful reasoning amongst those it recruits
• People receive lots of social/material benefits for continuing to believe that AI risk is the most important thing and work on it
• Even under some of the shortest timelines and most speculative capabilities that superintelligence might have, I really don’t extinction within the next year should be assigned any notable probability mass, due to the need for data centers to be replenished

I also see a lot of disagreements that I think are poorly reasoned. For example: 

• Poor application of/thinking about EV when conceptualizing risks
• Psychoanalysis that doesn’t explain the safety community’s behaviors well
• Comments significantly overstating the case for something
• Arguments without the slightest amount of research into what the other side actually believes

Tbh though, most of these critiques apply to both sides.

I think this primarily arises because the people working in AI safety who are the most careful reasoners also have the highest opportunity costs, because they tend to be good at other things. This means it’s less valuable for them to write out their thoughts online carefully.

My main point, though, is that it’s a little scary to see so much going wrong with the discourse and not being able to do anything about it (or even be worth trying to do anything about it). It feels like there’s not much worth trying except praying for Forethought to do a good job and reach everyone.