Building the EA/AI Safety Scene in San Francisco

This is an entry for the Manifund Essay Prize. It was awarded second-place for the following question:

"The race to build AI is going on in SF. So why is the AI safety scene here so weak? Berkeley, London, and DC all offer examples to learn from, but SF has its own unique challenges and opportunities. Beyond that, ecosystems like the startup scene, movements like climate change, and even religions may offer lessons for how to proceed."

Epistemic status: Written hurriedly for a competition with a short submission window whilst I was rather sick. This shouldn't be taken as my position after deep reflection. Instead, this is just an attempt to sketch out what a coherent strategy might look like as a baseline.

• I want to focus primarily on AIS, secondarily on EA:
  • Is EA or AIS more important? In SF, I'd say definitely AIS.
  • Re EA: I'd suggest focusing on something like Will's EA in the Age of AGI:
    • My intuition is that EA isn't resonating as well as it used to. Officially (on a global level) numbers are up, but EA isn't drawing the same talent as it used it. This is unsurprising—movements resonate in a particular time and place—and circumstances change, they always do.
    • I agree there are downsides to focusing heavily on AI, but if the alternative is no EA-like scene is SF, or one that struggles (as previous attempts have), then these downsides end up as much less significant.
• Object-level Challenges (the first two are based on this post):
  • Organisers tend to be rapidly syphoned away into more impactful projects
  • Lots of competition - other events, private events
  • Different kinds of events attract different subsets of the crowd
  • Regarding EA: Doesn't seem to  resonate as much as it did before
• Three Principles for responding to these challenges:
  • More Dakka: it feels plausible to me that prior attempts have failed due to insufficient resources
  • Adapt to Current and Local Circumstances
  • Think Outside of the Box—don't just do what works in other cities with different circumstances. That makes more sense in other cities, but not in SF.
• Concrete Instantiation:
  • More Dakka requires getting a funder on-board. You need to demonstrate the unique importance of SF:
    • Succeeding here will get you the resources you need to have an impact and to make field-building opportunities more attractive to the required talent. How money helps:
      • You can pay your field-builders more competitively; this is especially important when comparative opportunities pay more (such as in SF). Given SF's importance, this expenditure can be justified.
      • However, the best candidates are less focused on money than other folks. You can convince them to work for less if they believe it'll make a difference. This doesn't negate the need for money— they still need  to be properly resourced so they can run more ambitious projects.
      • It allows you to provide benefits apart from directly increasing salaries. More money means a longer runway and less stress. It also allows you to provide anciliary benefits like the ability to attend overseas conferences and compute for experiments.
      • It allows specialisation via a bigger team. This is possible no matter what the focus is, but this is especially challenging for building a broad community. This involves an exceptionally broad range of skills: technical knowledge, governance knowledge, communication, marketing, management, fundraising, career advising, ops, social skills, strategy... No-one's going to have it all. Distributing responsibilities makes it less stressful and reduces the chance that you're spending a significant amount of time on tasks that you hate.
    • Case for investing hard in SF:
      • SF has a unique density of talent and that talent has a unique ability to transition compared to other cities.
      • EA/AIS is tilted to thinking on a global scale. I think we've failed to appreciate that some cities are much more important than others. Concentrated talent is more important than talent spread out (see Cities and Ambition)
      • In most cities we primarily care about talent acquisition, in SF we also care about general attitudes about AI. There's value in exposing people to these ideas even if they don't go on to pursue AIS because the values in the watersupply in SF will impact how AI is built.
  • Adapt to current and local circumstances:
    • Regarding strands of activities:
      • More strands of activities is better so long as you have the staff to support running them well
      • However, if you don't have the staff to support running them well, you want to run less strands of activities. What makes it tricky is that's it's tempting to just try squeezing in one more thing.
      • You really want to convince the funders to provide sufficient funding for you to run multiple strands of activities as there are significant synergies from running multiple strands of activities due to cross-referral of participants.
    • SF Mass Outreach TOC:
      • Given the importance of what values are in the water supply, someone really should be pursuing this strategy
      • Comparison with typical AIS city work: Building a local *unified community* appears uniquely hard given the scene, whilst larger events appear uniquely valuable given the value of shifting baseline SF views, so I'd suggest making these larger events the *first* priority:
        • The global AIS scene is already drawing massive amounts of talent (fellowship applications doubling each year). SF has a lot of talent, but the value of this talent is subject to decreasing marginal returns. Further, the best talent in terms of current skills will already be competitive for jobs or selective fellowship. People with potential and agency can likely gain the experience they need through informal opportunities first.
      • Re EA: I'd suggest focusing on something like Will's EA in the age of AGI, as discussed above:
        • For mass outreach, having a broader range of topics is good, as prevents things getting old whilst allowing you to keep discussion at an introductory level.
        • SF viewpoints will affect a lot of issues, not just core AIS
        • This seems to suggest it might make sense to even merge EA and AIS fieldbuilding in the Bay = more concentrated resources, though depends on funder willingness to pay.
    • Strand 1 - Mass outreach events:
      • Why?
        • SF has uniquely good access to speakers whom people would *actually* go out of their way to hear (which is much better than having speakers just to have a speaker). Even if they aren't based there, SF is more likely to attract an audience, so they're more likely to be willing to travel.
        • Free food is a time-tested way of drawing more folks to talks. This is more likely to pass a cost-benefit analysis in SF than elsewhere (though, more generally, I expect this to become more common with funding becoming more available again)
        • Comparison to next strand: The next strand is a newsletter or Discord; these aren't first because they are much more valuable when centered around a community (even if it's only a community in a vague sense, rather than a community that builds deep connections). The strand after that is conferences, this niche is already filled to some extent, so it doesn't make sense to prioritise it higher
      • Structure:
        • For mass outreach events, presentation is key. This is different from more niche events where presentation doesn't have to be quite as smooth since folk are appreciative of having any kind of event at all. You want it to be professional quality, both for the sake of the audience and prospective speakers.
        • You almost certainly want a dedicated marketing person (even if it's part-time) and marketing budget. If you're spending the money on producing high-quality events, you want to be marketing it properly. One additional benefit: marketing these events increases the baseline saliency of AIS, even if someone never attends a talk. They're still more likely to Google it, have conversations about it or apply for other opportunities. High quality professional marketing with prominent speakers creates social proof.
    • If you want to add a second strand of activities, I'd suggest creating a proper newsletter/social media presence/and an online community (like a Slack or Discord). By proper, I mean "not purely focused on direct promotion". These synergise with the mass outreach strategy by scaling well:
      • The next strand I suggest is a conference, but these require a *lot* of work and capability to run, especially compared to this strand which can be fairly light.
      • Doing these well serves as a form of promotion for your events, even when a post/conversation/newsletter segment isn't directly engaging in promotion of your events. They still shape your audience's relationship towards you.
      • The nice thing about a newsletter is that you don't have to personally create the opportunities for folks to take a next step. You just link them to opportunities that already exist.
      • You want to make signing up for the newsletter effortless. For example, whenever you create an event survey, also include a question for them to subscribe. Or QR codes on presentations or handouts ("oh people can just type it in, it's not that hard"; no, they won't, they really won't)
      • It's worth investing a decent amount of time in choosing the channels. Affordances really matter. For example, if you want the community to run their own events (do you, or will this just be the blind leading the blind?), then you should create a channel for it. Otherwise, it probably won't happen.
    • If you want to add a third strand of activities, I'd suggest running something like a conference. This fits in well with the mass outreach plan and it utilises the same skills. It allows people who've attended your individual events to go deeper and it also serves as a tentpole event to draw more people into your community such that they hopefully attend the individual events.
      • For EA, I'd suggest a local EA conference that's more accessible than EAG SF.
      • For AIS, you also want to avoid filling the same niche as EAG. I would suggest running a conference that tilts more introductory (the first day of the Australia AI Safety Forum did this well). You could go more niche/selective, but The Curve and FAR.AI already serve this audience.
      • This is likely to work esp. well in SF due to a convergence of shovel-ready talent and opportunities that doesn't exist to the same extent elsewhere. Given this, I strongly recommend including a careers fair as a way of shifting the possibility of working into a concrete possibility.
    • If you want to add a fourth strand of events, I'd suggest running a variant of the AIS fundamentals course (they don't let you use that name) or creating a new course based on EA in the age of AGI.:
      • Whilst this is more medium-outreach than mass outreach, it synergises well with the mass outreach strategy. Like once you've picked the low-hanging fruit in mass outreach, it makes sense to move down the chain.
      • Someone might argue that deeper knowledge only has significant valuable if they go on to pursue AIS - I'd be more inclined to agree outside of SF - but within SF my intuition is that this is valuable because it increases the diffusion of knowledge.
      • Some proportion of folks who attend the mass outreach talks will want to go deeper and courses function well as a next step:
        • You could imagine an intermediate step of more advanced talks, but these are largely unnecessary. People who want to learn more and who don't want a medium commitment have access to countless resources online. Course allow people who are open to a medium commitment to compress their learning (an 8 session reading group could compress what would otherwise take 1.5-2 years of attending an event every few months) and to figure out whether they want to make a large commitment (empirical observation, but makes sense if you think of commitment requiring a certain level of exposure for you to decide).
        • A yearly conference can only substitute if the timing is fortuitous; and even then, it won't go as deep or be as systematic as the course.
      • It's easy to think "oh there's a global BlueDot course", but this is worse from both an experience and a marketing perspective.
        • From an experience perspective, travel is frustrating, but there's a lot of benefit of meeting people in the same city. You're much more likely to keep in contact with them and to form collaborations.
        • Beware Trivial Inconveniences. It's much more natural to invite folks to join a course you're running than one someone else is running. You can also choose for the start time of the course to make sense based on local conditions - for example, I've organised two iterations of AIS courses timed to start shortly after EAGx Australia. I don't have a comparison, but I believe timing it like that was the right call.
    • If you want to add a fifth strand of events, I'd add advising calls. These could be career advising calls or they could be assisting AIS people new to SF:
      • This breaks with the overall mass outreach strategy at all—but it synergises nonetheless. By this stage we've likely picked the low hanging fruit there. You could try running meetups for more engaged members, but it is hard to get these working in SF.
      • That said, your mass outreach activities will probably convince some very high potential people become involved and, at this stage, these people are worth investing time in.
  • Think outside the box:
    • You can see where we broke existing assumptions. The main one is that in most other cities diffuse awareness building has limited value, but SF is different. If doing what worked in other cities worked, it would have already worked, so that's why I'm proposing something else.
• Limitations of this plan:
  • Whilst I've visited SF many times, I don't live there. That limits my understanding of the local context.
  • If AI timelines are super short, then my TOC may not have time to play out. I am quite worried about the "great plan... for 2022" objection.
  • Mass outreach events require a high degree of professionalism. This limits your ability to use grads right out of uni to run the activities.
  • My plan assumes that funders can be persuaded to invest more money. This might not be the case, especially if field-building attempts have repeatedly failed in the past. However, I don't view the willingness of funders to spend as fixed. Instead, I expect this depends on presenting a persuasive plan and building the right team.