Andrew Snyder-Beattie and Ethan Alley
This is a list of longtermist biosecurity projects. We think most of them could reduce catastrophic biorisk by more than 1% or so on the current margin (in relative terms). While we are confident there is important work to be done within each of these areas, our confidence in specific pathways varies widely and the particulars of each idea have not been investigated thoroughly.
Still, we see these areas as critical parts of biosecurity infrastructure, and would like to see progress building them out. If you’d like to be kept up to date about opportunities to get involved, please fill out this Google Form.
Early Detection Center
Early detection of a biothreat increases the amount of time we have to respond (e.g. designing tailored countermeasures, using protective equipment, heading to bunkers, etc). The current approach for early warning of novel pathogens is severely lacking—it typically relies on a particularly astute doctor realizing that something is strange combined with negative tests for everything else. Existing systems are also almost exclusively focused on known pathogens, and we could do a lot better using pathogen-agnostic systems that can pick up unknown pathogens.
One concrete goal would be something simple where a small team of people collects samples from volunteer travelers around the world and then does a full metagenomic scan for anything that could be dangerous. Even collecting and analyzing only 100 random samples per day could make a big difference in some scenarios, since that would mean we would still expect to catch things before they infect too large a fraction of the global population. We think that with the right team, this could be done with close-to-existing technology for less than $50 million per year.
There are a handful of bottlenecks and a number of ways to decompose this problem. To get started on subproblems, one of us (Ethan) is working on a list of suggestions, which we will backlink here.
Super PPE
Most personal protective equipment (PPE) is not good enough. Things like masks and suits require training to fit properly, lack reusability, and are generally designed for routine uses rather than for the most extreme events. The small minority of PPE that is designed for extreme use cases (e.g. BSL4 suits or military-grade PPE) is bulky, highly restrictive, and insufficiently abundant—not the kind of thing you could easily put millions of healthcare/pharma/essential workers into if needed. It seems plausible that with good materials science and product design we could come up with next-generation PPE that is simultaneously highly effective in extreme cases, easy to use, reliable over long periods of time, and cheap/abundant.
One concrete commercial goal would be to produce a suit (and accompanying system) that is designed for severely immunocompromised people to lead relatively normal lives, at a cost low enough to convince the US government to acquire 100 million units for the Strategic National Stockpile. Another goal would be for the suit to simultaneously meet military-grade specifications, e.g. protecting against a direct hit of anthrax.
PPE has the advantage of being truly ‘pathogen-agnostic’—we can stockpile it in advance of knowing what the threat is, in contrast to vaccines or many medical countermeasures. It is also ‘defensively stable’ in that physical barriers can’t be easily bypassed using pathogen engineering techniques (whereas many medical countermeasures might be defeated with some creative tinkering). See Carl Shulman’s post here for more on this.
To get started on subproblems within PPE, one of us (Ethan) will publish a PPE deeper dive at some point in the future (backlink forthcoming).
Medical countermeasures
Medical countermeasures (e.g. vaccines, antivirals, monoclonal antibodies) for use against catastrophic biorisks have a number of existing drawbacks. In most cases they are tailored to existing pathogens (e.g. smallpox vaccines) and wouldn’t help against a novel threat. Many countermeasures are also not robust against deliberate engineering (e.g. antibiotics are broad-spectrum but can be overcome).
We think there could eventually be opportunities for radically improved medical countermeasures against GCBR-class threats, either by 1) producing targeted countermeasures against particularly concerning threats (or broad-spectrum countermeasures against a class of threats), or by 2) creating rapid response platforms that are reliable even against deliberate adversaries.
However, we are not yet ready to recommend medical countermeasures as a general focus area for large scale projects, in part because many projects in this space have inadvertent downside risks (for example, platforms that use viral vectors may accelerate viral engineering technology). If you feel excited about working in this area, fill out the Google Form (here) and we might be able to provide you some more tailored advice.
BWC Strengthening
Right now, the biological weapons convention (BWC)—the international treaty that bans biological weapons—is staffed by just four people and lacks any form of verification. We think there is more scope for creative ways of strengthening the treaty (e.g. whistleblowing prizes), or creating new bilateral agreements and avoiding bureaucratic gridlock. Moreover, a team of people scouring open sources (i.e. publication records, job specs, equipment supply chains) could potentially make it difficult for a lab to get away with doing bad research, and thereby strengthen the treaty.
Sterilization technology
Sterilization techniques that rely on physical principles (e.g. ionizing radiation) or broadly antiseptic properties (e.g., hydrogen peroxide, bleach) rather than molecular details (e.g. gram-negative antibiotics) have the advantage of being broadly applicable, difficult to engineer around, and having little dual-use downside potential.
Existing technologies for physical sterilization (e.g. UV light, materials science for antimicrobial surfaces, etc.) have different limitations in terms of costs, convenience, and practicality, and we think this is an underexplored area for prevention and countermeasure development. We have a lot of remaining uncertainties in this area but think the value of investigating it is high.
Refuges
Existing bunkers provide a fair amount of protection, but we think there could be room for specially designed refuges that safeguard against catastrophic pandemics (e.g. cycling teams of people in and out with extensive pathogen agnostic testing, adding a ‘civilization-reboot package’, and possibly even having the capability to develop and deploy biological countermeasures from the protected space). This way, some portion of the human population is always effectively in preemptive quarantine.
Another way of framing this: lots of people think we’d substantially reduce biorisk if we had a self-sustaining settlement on Mars (and we basically agree). If that’s the case, it would be a lot cheaper to put the exact same infrastructure on Earth, and it buys almost the same amount of protection.
One next step on this would be to build an org which specializes in the operations, logistics and contractor relationships needed to actually build a refuge with the necessary amenities (based on a shallow investigation, one of us, ASB, ballparked outsourcing at around $100-300M / bunker but did not have logistics expertise or time to dig deeper). We have some more ideas in the works we’ll backlink here later, but please fill out the form if you’re interested in the meantime.
Conclusions
A few things we want to highlight:
- Collectively, these projects can potentially absorb a lot of aligned engineering and executive talent, and a lot of money. Executive talent might be the biggest constraint, as it’s needed for effective deployment of other talent and resources.
- Many of the most promising interventions are not bottlenecked by technical expertise in biology or bioengineering. Technically minded EAs who want to work in bio should consider training in other areas of engineering, and in general look to build generalist engineering and problem-solving skills rather than focusing only on getting biology knowledge.
- These projects have reasonably good feedback loops (at least compared to most longtermist interventions), making this area a promising proving ground for meta-EA interventions, especially around entrepreneurship.
Despite how promising and scalable we think some biosecurity interventions are, we don’t necessarily think that biosecurity should grow to be a substantially larger fraction of longtermist effort than it is currently. From a purely longtermist perspective we think that AI might be between 10-100x more important than biosecurity, even if solving biosecurity might be more tractable than solving AI (possibly by a large factor). Biosecurity is also attractive as a cause area for non-longtermist reasons, given the importance of preventing lesser catastrophes that fall short of truly civilization-ending but are still horrific (e.g., 10-100x COVID)—we thus think it could be even more relatively appealing for those more focused on impacts on the current generation(s).
Again, please fill in this coordination form to stay informed of developments and opportunities.
We thank Chris Bakerlee, Jamie Balsillie, Kevin Esvelt, Kyle Fish, Cate Hall, Holden Karnofsky, Grigory Khimulya, Mike Levine, and Carl Shulman for feedback on this post.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Really happy to see this, this looks great.
This is outside the scope of this document, but I'm a bit curious how useful it would have been to have such a list 3-5 years ago, and why it took so long. Previously I heard something like, "biosecurity is filled with info-hazards, so we can't have many people in it yet."
Anyway, it makes a lot of sense to me that we have pretty safe intervention options after all, and I'm happy to see lists being created and acted upon.
The authors will have a more-informed answer, but my understanding is that part of the answer is "some 'disentanglement' work needed to be done w.r.t. biosecurity for x-risk reduction (as opposed to biosecurity for lower-stakes scenarios)."
I mention this so that I can bemoan the fact that I think we don't have a similar list of large-scale, clearly-net-positive projects for the purpose of AI x-risk reduction, in part because (I think) the AI situation is more confusing and requires more and harder disentanglement work (some notes on this here and here). The Open Phil "worldview investigations" team (among others) is working on such disentanglement research for AI x-risk reduction and I would like to see more people tackle this strategic clarity bottleneck, ideally in close communication with folks who have experience with relatively deep, thorough investigations of this type (a la Bio Anchors and other Open Phil worldview investigation reports) and in close communication with folks who will use greater strategic clarity to take large actions.
I have only been involved in biosecurity for 1.5 years, but the focus on purely defensive projects (sterilization, refuges, some sequencing tech) feels relatively recent. It's a lot less risky to openly talk about those than about technologies like antivirals or vaccines.
I'm happy to see this shift, as concrete lists like this will likely motivate more people to enter the space.
More than infohazards, we were still building capacity and understanding of the area.
But many of these were highlighted in earlier work, including decade of reports from Center for Health Security, etc. (Not to mention my paper with Dave Denkenberger.)
thanks for the kind words! I agree that we didn't have much good stuff for ppl to do 4 yrs ago when i started in bio but don't feel like my model matches yours regarding why.
But I'm also wanting to confirm I've understood what you are looking for before I ramble.
How much would you agree with this description of what I could imagine filling in from what you said re 'why it took so long':
"well I looked at this list of projects, and it didn't seem all that non-obvious to me, and so the default explanation of 'it just took a long time to work out these projects' doesn't seem to answer the question"
(TBC, I think this would be a very reasonable read of the piece, and I'm not interpreting your question to be critical tho also obviously fine if it is hahah)
That sounds like much of it. To be clear, it's not that the list is obvious, but more that it seems fairly obvious that a similar list was possible. It seemed pretty clear to me a few years ago that there must be some reasonable lists of non-info-hazard countermeasures that we could work on, for general-purpose bio safety. I didn't have these particular measures in mind, but figured that roughly similar ones would be viable.
Another part of my view is,
"Could we have hired a few people to work full-time coming up with a list about this good, a few years earlier?"
I know a few people who were discouraged from working in the field earlier on because their was neither the list, nor the go-ahead to try to make a list.
I don't think any of the info hazards are mentioned here, but you're right that good lists like this are a long time coming. I haven't heard that biosec folks actively didn't want people in the field though-- would be interested in who said that.
FWIW, I know of a case from just last month where an EA biosecurity person I respect indicated that they or various people they knew had substantial concerns about the possibility of other researchers (who are known to be EA-aligned and are respected by various longtermist stakeholders) entering the space, due to infohazard concerns.
(I'm not saying I think these people should've been concerned or shouldn't have been. I'm also not saying these people would have confidently overall opposed these researchers entering the space. I'm just registering a data point.)
I am surprised, and feel like I need more context. "This space" is probably too vague. I'm definitely opposed to even well-aligned people spending time thinking up new biothreats. But that's very different than working on specific risk mitigation projects.
By "this space", I meant the longtermist biosecurity/biorisk space. As far as I'm aware, the concern was along the lines of "These new people might not be sufficiently cautious about infohazards, so them thinking more about this area in general could be bad", rather than it being tailored to specific projects/areas/focuses the new people might have (and in particular, it wasn't because the people proposed thinking up new biothreats).
(But I acknowledge that this remains vague, and also this is essentially second-hand info, so people probably shouldn't update strongly in light of it.)
I would agree that getting people who aren't cautious about things like infohazards is a much more mixed blessing if we're talking about biorisk generally, and I'd want to hear details about what they were doing, and why there were concerns. (I can think of several people whos contribution is net-negative because most of what they do is at best useless, and they create work for others to respond to.)
But as I said, the pitch here from ASB and Ethan was far more narrow, and mostly avoids those concerns.
It seems reasonable to me to be vigilant of sharing infohazards with new researchers in the field. Still, I am wondering if it might actually be worse to leave new researchers in the dark without teaching them how to recognize and contain those infohazards, especially when some are accessible on the internet. Is this a legitimate concern?