One concrete goal would be something simple where a small team of people collects samples from volunteer travelers around the world and then does a full metagenomic scan for anything that could be dangerous.
Could you feasibly get the same information from airport waste collection? I'm thinking of an airport/travel focused BioBot, but with pathogen agnostic ambitions.
One interesting and somewhat-related story here: an airport spa chain called XPresSpa launched a COVID-testing service called XpresCheck and have been working alongside Concentric by Ginkgo on airport biosurveillance for specific countries:
“For the past 3 months, XpresCheck and their testing partner, Concentric by Gingko, demonstrated that a traveler-based SARS-CoV-2 viral genomic surveillance program can help detect emerging SARS-CoV-2 variants in the United States,” said Dr. Cindy R. Friedman, chief of CDC’s Travelers’ Health Branch. “In fact, following the rapid expansion over the Thanksgiving break, the program detected the first documented U.S. cases of Omicron sub-lineage BA.2 and BA.3,” added Friedman.
I think Concentric is aiming to continue running airport biosurveillance; the idea of working from airport waste (rather than needing to directly sample travellers) is super interesting!
Potentially - this is something myself and others working on metagenomic monitoring have discussed and would like to investigate the practicalities of. If anyone has connections to international airlines or knows about the legalities/ownership of airline waste, I'd be interested in chatting.
My company has a few airline and airport clients. My understanding is that waste management is a service the airport provides to the airlines, and I'd guess there are third party contractors involved.
I'll be on the lookout for opportunities to learn more, and report back if I hear anything useful.
Great!
My take is that these projects are not super information hazardy and what could be useful is for a subset of these projects (super PPE/sterilization?/countermeasures?) is a bunch of smart, entrepreneurial people who are reasonably aligned who can spend a few months bashing out ideas in a lab/office space somewhere.
I think that iGem is pretty entrepreneurial, it tends to attract a bunch of smart, motivated people who are psyched about working on synthetic biology (at least in my limited experience). Competitors tend to design projects around tracks and I could imagine $100k could buy influence to set up a track or (launch some other programme) to get ~400 working on these problems for 3 months (6000 competitors/(13 tracks + 'our preventing pandemics' track)). This works out at ~$250 per 3 person-months work which seems pretty reasonable. I could imagine it being an order of magnitude worse but it still seems like a good bet.
I'd expect for something like 80 projects to be investigated and 5 of them to be useful. There seems to be little downside counterfactual harm wise as they would be doing synbio stuff anyway. It's plausible to me that directing them towards biosec projects has bad worst-case scenarios relative to synbio - I have thought very little about this, but I'd be surprised if these particularly bad projects weren't caught by their biosec team.
iGem even seems to care quite a lot about biosecurity and I know that there are some links between open phil and the iGem biosec team so it's plausibly even low cost to investigate.
Hi, I work on biosecurity at iGEM, can confirm we care quite a lot about it. A lot of these projects don't seem obviously best solved through synthetic biology (cf. Biosecurity needs engineers and materials scientists) but iGEMers often surprise me!
I think many teams are already motivated to work on medical countermeasures, though I maybe see a somewhat greater number of exciting diagnostics projects than therapeutics projects (for example, two of the winners of the 2020 competition worked on rapid point-of-care diagnostics (https://2020.igem.org/Team:Leiden/Description, https://2020.igem.org/Team:TAS_Taipei). I would guess this is because it's easier to measure the success of a prototype diagnostic over a few months than it is to figure out a relevant assay for treating a disease.
Last year we tried to incentivize more direct work on technical advances in biosecurity via giving out 5 microgrants (https://2021.igem.org/Teams/Grants/Safety) and doing more to promote and spotlight our award for Safety and Security (https://video.igem.org/w/nkrCA4EaFGEuefUtbmLijN). We'll be iterating on that program this year, though I don't know exactly what form it will take; I'm definitely taking inspiration from the ideas here and in the Future Fund ideas thread, though.
Because futures are associated with specific dates, the relative prices of different futures can give you information about when in the future PPE is likely to be scarce or plentiful. In contrast, the valuation of a company just tells you one number that represents the discounted sum of all future profits. Additionally, manufacturers can sell futures to de-risk investment/production plans.
[Epistemic status - I have no idea what I'm talking about.]
Could you set up a futures exchange, with physical-delivery contracts, around various PPE types? From my extremely naive understanding, couldn't this direct more capital available towards PPE production, plus as an ancillary benefit act as an implicit prediction market of pandemic risk?
I haven't thought this all the way through, but the potential to push more capital toward production, transparently allocate supply, and also get a warning signal seems quite appealing on first blush.
I went to this site and attempted to click on the Non-trivial privacy policy: https://archive.non-trivial.org/privacy-policy and got the "this site can't be reached" message.
There is a moral hazard in treating refuges—whether they are high-tech bunkers or colonies in Mars/Moon—as one of the biosecurity projects to address existential risk. We risk decoupling the interests of the global elite from the survival of the general population. If the wealthy can afford a 'way out,' the collective incentive to solve existential threats on Earth from engineered pandemics or AI misalignment is weakened. We should be investing in 'public good' safety measures, not private lifeboats.
I'm quite fascinated by this post because I work for a company that spent a chunk of its startup years trying to implement the "Early Detection Center" part using 911 calls and call-related data.
From listening to the early folks, I got the impression that "terrorism! biosurveillance!" made for nice press conferences. But, people are mostly interested only if we help their highly-visible and much more obvious key performance indicators improve (e.g. increasing revenue). Even after getting certified (?) by the US Department of Homeland Security as a syndromic surveillance system, we continue to spend almost all our time on everything everything but syndromic surveillance.
As a publicly visible example of the company's change, you can go to https://firstwatch.net/what-we-do/ and see that part of our business is only 1 tile ("Public Health") out of 5.
Maybe someone can learn from our experiences and find a way to persuade people to take it seriously in the future.
This is fascinating.
I find it easy to slot this into my worldview that for profit mixes very poorly with altruism (or just out-replicates virtuous ideas in internal environments mundanely).
But also this just seems a good pivot?
I’m confused about one thing: it seems like you present a service for government customers on your website. But these customers don’t maximize revenue, right?
Maybe what you’re saying is that dealing with these public entities is just one facet of the business—or you even buy data or operate at a loss from them even. The data you collect in that service is used to support a business with customers who are private entities?
I’m sorry that the altruistic and virtuous mission of identifying major risks isn’t interesting in the actual business and day to day. This seems like a sign it is neglected.
EDIT: I read a bit more and I think the customers you’re talking about is EMS and hospitals. Which would maximize revenue (which isn’t obvious to non Americans).
You're correct except that we receive money from other types of organizations too, including non-profit organizations who give money in the form of grants (hi there, American Heart Association!). You'll see why later in this comment.
The firm has international ambitions but it is an American company with an office in California.
>buy data at a loss?
Not quite that cheap. You can think of it as 'Insert coins. Get a table of data about people in trouble.' More specifically, we charge people for each data source they want us to look at.
The most common type of permission ("HIPAA Business Associates Agreement") doesn't let us share information with other people. I'm fairly certain FirstWatch requires additional legalese before showing 1 organization's bits to another organization.
For more details about payments and how things work, you can look at pages 6 and 7 of "FirstWatch Agreement 2011" in a customer's agenda item.
Since the "revenue maximization" part caused trouble, I'll explain further. If you live in the USA, you can stop here.
Imagine the following chain of events:
There's going to be a wait time measured in minutes or hours before each step from step 4-8.
Even if an organization wanted to watch for a disease outbreak, make sure to give people the right medical treatment, and let employees have a good life, it has to watch out for $.
Each organization will send a bill to you, your health insurance company, the people who collect your taxes, or some combination of the above. And the bill must be backed up by documentation.
Example:
Step 4:
Step 5:
Step 6: Some firefighters employed by "Fire Alarm Boxes, Inc." or "Fire Alarm Boxes in South-Mega Region Joint Powers Authority"
Step 7: Ambulance crew employed by either "Almost Bankrupt Ambulance Company, Inc.", "Gigantic Group of Ambulances, Inc.", or "City of Little Town". City of Little Town is next to City of Big Name Here.
Step 8:
If you're wondering when the Public Health Department gets involved, the answer is "never" unless you write a law that says "thou shalt report cases of X or else."
Thanks for laying this out, it's very interesting and shows a lot of depth of knowledge. Each of those steps is important.
My sense is that you're providing support and accounting for these entities for billing essentially.
What you wrote also gives a sense why there's not much "ownership" or attention to pandemic safety or genomic sampling. Honestly, it looks like things are barely held together ("Almost Bankrupt Ambulance Company").
You're correct except that we receive money from other types of organizations too, including non-profit organizations who give money in the form of grants (hi there, American Heart Association!). You'll see why later in this comment.
I think you're saying that without this funding, there would be pretty much no support for any social goods that you can provide.
without this funding
The original post by the two authors talked about getting effective monitoring/surveillance.
The status quo is what the planet has with the current funding, etc. If you want something better, then as you inferred, it's going to take additional changes and resources.
In the Less Wrong sequences, there are essays about utilions and warm fuzzies. In the healthcare world, that distinction is always present.
If I spend $80,000 hiring someone to slog through medical records looking for a pandemic, then I have given up the chance to spend $80,000 on a nurse to get patients out of ambulances and into a hospital bed faster. The former cannot be billed to Medicare. The latter can be.
To use an AI analogy, if a programmer makes a reward function that rewards the latter and not the former, the programmer doesn't get to complain that it was a surprise while dying of COVID-19 or being turned into a paperclip.
Everything you said makes sense to me and seems wise. Please continue if you have more wisdom to share.
Just so you know, I'm basically a random person on the internet, but I want to point you in any directions if I can help.
As a random person, are you looking for funding or support? What are the best outcomes you want from your presence on the forum?
>what do you want?
Mostly wanted to describe what has been tried before so that maybe someone else can try something smarter in the future. There's so many misaligned incentives and problems that it's hard to know where to start and it's nice to have a place to put these thoughts down in a productive manner.
I guess I was looking for emotional support and got it; thank you. I imagine that my emotions have similarities to that of people who work on friendly AI.
There's not much else to say that isn't said better elsewhere (woes of American healthcare, coordination problems, the LW sequences, the way brains think of other people, utilions, heroic responsibility, ambiguous delegation of duties, dissemination and usage of knowledge, etc.).
In closing, a passage from chapter 109 of HPMOR feels appropriate.
The aspect I found interesting was that ... the rest of Atlantis ignored this project and went upon their ways. It was sometimes praised as a noble public endeavor, but nearly all other Atlanteans found more important things to do on any given day than help. ... With relatively little support, the tiny handful of would-be makers of this device labored under working conditions that were not so much dramatically arduous, as pointlessly annoying. Eventually time ran out and Atlantis was destroyed with the device still far from complete.
Thank you for sharing!
(I don't really know how to operationalize this, I'm not connected to the people working on pandemic prevention) but I hope your experience and interests find use where it's helpful, and you find it satisfying and rewarding to contribute.
Despite how promising and scalable we think some biosecurity interventions are, we don’t necessarily think that biosecurity should grow to be a substantially larger fraction of longtermist effort than it is currently.
Agreed that it shouldn't grow substantially, but ~doubling the share of highly-engaged EAs working on biosecurity feels reasonable to me.
FWIW, I don't actually know what you mean/believe here and whether it's different to what the post already said, because:
I do mean EAs with a longtermist focus. While writing about highly-engaged EAs, I had Benjamin Todd's EAG talk in mind, in which he pointed out that only around 4% of highly-engaged EAs are working in bio.
And thanks for pointing out I should be more precise. To qualify my statement, I'm 75% confident that this should happen.
Why are suits and substances used to sterilize surfaces (e.g., hydrogen peroxide, bleach) mentioned in relation stopping pandemics? Another post by one of the authors (ASB) of the current post mentioned a self-sterilizing suit regarding the same subject.
Suits and surface sterilization seems unnecessary, because that stuff does nothing to stop airborne transmission of viruses, which seems to be the only way that pandemics can ever arise.
Airborne transmission of respiratory viruses
https://doi.org/10.1126/science.abd9149
Just want to note that there are, in fact, 2 ongoing pandemics, and the earlier one, AIDS, isn't a respiratory virus. And those two obviously don't comprise an exhaustive list of possible transmission vectors.
AIDS is considered to be an epidemic, not a pandemic, but can a sexually-transmitted disease similar to AIDS lead to a pandemic? I doubt it, because pandemics are dangerous (in part) due to rapid spread, a feature which a sexually-transmitted disease will never possess. I'd be a slightly more worried only if everyone was a lot more promiscuous.
Yes, respiratory diseases are far and away the most likely causes of GCBRs, but even there there is risk from other types of disease. But we're discussing pandemics, and there you seem a bit misinformed.
First, the WHO has recently classified AIDS as an endemic disease, changing from its earlier classification as a pandemic. (Just like we'll do with COVID-19 in a couple years.) But that didn't make it not a pandemic until that point. And not only has AIDS killed 35m+ people - easily twice the total for COVID-19, but somewhere between 500k-1m more people were killed this year. And absent AI or a biotech solution to AIDS, that is likely to continue for several more decades. Making this even worse, unlike COVID-19, which skews towards killing people in poor health and the elderly, they were almost all people who would otherwise have lived far longer and healthy lives. If AIDS is not they type of thing we want to stop when we say we're hoping to end pandemics, I don't know what is.
Second, in the past, in addition to AIDS being a clear example of a STD pandemic, fecal-oral spread diseases have caused pandemics, as have vector borne diseases. (And not only that, but many weren't viruses - and antibiotic resistance should worry us on that front. And outside of viruses and bacteria, Plasmodium kills millions a year.) So again, the general point was that it's simply not the case that airborne transmission of viruses is the only way for pandemics to arise.
This discussion is about preventing and mitigating pandemics that could potentially end civilization, and stuff similar to AIDS (regardless of how you want to categorize it) is off topic because transmission would not be rapid enough to end civilization.
I don't think accusations of off-topic-ness at this point are very helpful.
You have been making strong claims about "pandemics" in general, which others have responded to by pointing out examples of pandemics that don't fit your claims. If by "pandemics" you meant "civilisation-ending pandemics" only, I think it was on you to make that clear.
In the context of this discussion (the post is about GCBRs), it should have been clear what I meant by that term.
Also, it can be claimed that a lot of things are "pandemics" like TB and antibiotic-resistant bacteria, but what is usually meant by the term is rapid, global spread (within weeks to a few months at most) of a deadly pathogen.
It wasn't obvious to me, and apparently also not to others, that your statements about "pandemics" were not meant to apply to pandemics in general.
In general, when you realise you have been communicating unclearly, it's a bad idea to blame the people you confused.
But obviously, there are transmission modes other than airborne and sexual, so I think you are missing my point about too-narrow thinking.
Because your claim is, effectively, a near-certainty that only airborne transmission could be threatening. And my response was, in effect, that this isn't correct, and that even eliminating airborne disease transmission completely wouldn't sufficiently address risks of future bioengineered pandemics, even if it would greatly reduce the number of viable such cases.
Why do you think fomite transmission is still worth considering?
I think you're once again focusing far too narrowly - foodborne illness, waterborne disease, intermediate animal hosts and parasites, vector borne diseases, fomites, and sexual transmission are all mechanisms that currently spread disease, and it seems very strange to say that we should only ever look at aerosol transmission. Yes, it's the most worrying, but it's not enough on its own to address all threats.
I think he was explicitly addressing your question of sexually-transmitted diseases being capable of triggering pandemics, not if they can end civilization.
Discussing the latter in detail would quickly get into infohazards—but I think we should spend some of our efforts (10%) on defending against non-respiratory viruses. But I haven't thought about this in detail.
The AIDs epidemic is widely considered a pandemic (pandemics are a subset of epidemics). And one of the deadliest pandemics of the 20th century, at that.
In the 19th century, cholera, a faecal-oral pathogen, caused several pandemics, killing very many people. It doesn't do that any more thanks to sanitation in rich countries, but it's certainly not impossible for non-respiratory pathogens to achieve rapid global spread.
Everyone agrees with you that respiratory viruses are the biggest concern, and you've provided some good resources in this thread that I appreciate. But I do think you are being undernuanced and overconfident here.
In order for us to be safe from future pandemics, it's really important we don't overindex on the pandemics of the past (or the present).
SARS-CoV-2 doesn't really spread through surfaces/fomites much, if at all.
I'm sure the linked post is right to say that this is also true of "several respiratory pathogens".
However I'd be surprised if it were true of all respiratory pathogens, let alone other diseases. Gastric/diarrhoeal diseases such as norovirus, rotavirus, or, indeed, ebola can spread through fomite transmission.
In short, I disagree that airborne transmission of viruses is the only way that pandemics can ever arise.
Why would you be surprised if airborne transmission was the only way that any respiratory pathogen could cause a pandemic?
I haven't seen any strong empirical evidence that fomite transmission is even a thing and mechanistic reasons to doubt that it could cause a pandemic even if it were a thing. My mechanistic reasoning is this: fomite transmission would be too convoluted (e.g., nose → hand → variable period of time → door knob → variable period of time → hand → variable period of time→ nose) to be compatible with the sustained and rapid spread necessary to cause a pandemic.
Gastric/diarrhoeal diseases don't and can't cause pandemics for mechanistic reasons: it's hard to infect people with bodily fluids.
I can't find the source anymore but I remember being fairly convinced (70%?) that rhinovirus is probably spread primarily via formites, fwiw.
The main thing is that snot can carry a lot more viruses than aerosols. It's also suggestive to me that covid restrictions often had major effects on influenza and RSV, but probably much less so on rhinoviruses.
I also don't think we should necessarily overindex on viral respiratory diseases/pandemics, even though I agree they're the scariest.
rhinovirus is probably spread primarily via formites
Until the COVID19 pandemic, nearly everyone thought that most infectious respiratory diseases were transmitted via fomites and droplets, but unfortunately, this was based on shockingly poor evidence and assumptions. The material you've seen is based on this outdated consensus.
As I pointed out before, there are mechanistic reasons to doubt that pandemics can arise from fomite transmission.
However, if I squint hard enough, I can kinda, sorta see how young children in daycare might be infected by sharing toys and sticking their fingers up their noses. But stuff like that isn't going to cause a pandemic.
In fact, the dominant (and most likely only) mode of transmission of rhinovirus is aerosols (at least in adults), not fomites. The same paper claims that fomites were unable to infect adults.
I also don't think we should necessarily overindex on viral respiratory diseases/pandemics, even though I agree they're the scariest.
Anything that's capable of causing a civilization-ending pandemic must be able to rapidly replicate in humans and spread via airborne transmission, and the only thing that can do that is viruses (and perhaps virus-like particles in the future).
The bottom line for me about fomites and surface contamination is that it's probably a non-issue, and even if there's something to it (which doesn't seem too likely), it's pretty easy to deal with without fancy tech: just wash your hands, disinfect surfaces, and wear a face covering to prevent face-touching.
That Wired article is fantastic. I see this threshold of 5 microns all over the place and it turns out to be completely false and based on a historical accident. It's crazy how once a couple authorities define the official knowledge (in this case, the first few scientists and public health bodies to look at Ward's paper), it can last for generations with zero critical engagement and cause maybe thousands of deaths.
I'm confused about the distinction between fomite and droplet transmission. Is droplet transmission a term reserved for all non-inhalation respiratory pathogen transmission (like touching a droplet on a surface and then touching your face, or the droplet landing on your mouth), so it includes some forms of fomite transmission? I'm seeing conflicting sources and a lot that mention the >5 μm rule so don’t seem too trustworthy.
cause maybe thousands of deaths.
Don't you mean millions of deaths?
I'm confused about the distinction between fomite and droplet transmission.
From what I've read, fomite transmission must involve surface touching, whereas droplet transmission must involve droplets, which are expelled by coughing or sneezing, directly landing (like a bullet) in your mouth, nose, or eyes without any extra contact or touching.
These methods of transmission seem so implausible (how many people actually sneeze or cough directly in someone's face?) to be major causes of spread that it's hard to believe that no one seemed to have performed definitive experiments to test these ideas for many decades. On the other hand, even seemingly definitive experiments (like the rhinovirus study) don't seem able to shift expert opinion. In the case of rhinovirus, maybe one experient isn't enough, but then the question is why no one seems to have been interested in replicating it.
Here's the study FYI.
Yeah, there were two groups that studied how rhinovirus is transmitted. One group was from the University of Virginia and found evidence of only fomite transmission. The study you cited is theirs. The other group was from the University of Wisconsin and found evidence for only airborne transmission. The Wisconsin group "...argued that the high rate of transmission via the hands in the Virginia experiments might be attributable to intensive contact with fresh wet secretions produced by volunteers who essentially blew their nose into their hand."
These projects have reasonably good feedback loops (at least compared to most longtermist interventions), making this area a promising proving ground for meta-EA interventions, especially around entrepreneurship. [emphasis added]
Do you mean that these projects would be a promising proving ground for people (esp. entrepreneurial types) who might want to later also do interventions seeking to build/strengthen the EA movement? If so, why?
I would've thought that (a) the biosecurity projects would provide similarly good or better feedback loops for other "object-level" entrepreneurial longtermist projects, and (b) entrepreneurial community building projects themselves would have good feedback loops so might be better proving grounds for further work of the same type?
Or maybe the idea is that meta-EA interventions aimed at getting more skilled longtermist aligned entrepreneurs could focus on funnelling them into doing the sorts of projects you propose in this post, and then how well that goes could be used to evaluated those meta-EA interventions?
meta note- its super cool to see all this activity! but the volume is makin me a bit stressed and i probably won't be trying to respond to lots even if i do one sporadically. does not mean i am ignoring you!
These approaches make sense as biosecurity interventions. As a biomedical engineer with biosecurity interests, I have a career interest in these questions. Here are a couple of my questions:
There are a handful of bottlenecks and a number of ways to decompose this problem. To get started on subproblems, one of us (Ethan) is working on a list of suggestions, which we will backlink here.
@eca, did you end up writing this list?
We think most of them could reduce catastrophic biorisk by more than 1% or so on the current margin (in relative[1] terms).
Imagine all six of these projects was implemented to a high standard. How robust do you think the world would be to catastrophic biorisk? Ie. how sufficient do you think this list of projects is?
Moreover, a team of people scouring open sources (i.e. publication records, job specs, equipment supply chains) could potentially make it difficult for a lab to get away with doing bad research, and thereby strengthen the [BWC] treaty.
Here's quite a good article calling for the same thing, which also outlines lots of promising avenues for open-source investigation (in addition to the above; trade data, patents, social media, satellite imagery, unusual epidemic reports, and environmental sampling):
Can everyone help verify the bioweapons convention? Perhaps, via open source monitoring
Paints it as an appropriately weighty task though, I quite liked this quote:
It’s something like searching for needles in 193 country-sized haystacks, each of which differs in size, structure, and complexity and might not contain any needles at all. The needles might not look like needles, either – they might exist only as needle components waiting to be put together
Thanks, this post seems super useful!
This version of a ‘sentinel system’ is going to be neglected by traditional public health authorities and governments because they won’t be searching for engineered threats designed to elude pathogen-specific detection tools
This sounds a bit too fatalistic to me. It does seem basically guaranteed that these authorities and governments will spend less attention and resources on this than longtermists would ideally like. I imagine you're also right that they'll very much neglect this, like building something far short of what longtermists would like. But that latter point doesn't seem ~guaranteed, and it seems like something we might have a real shot at changing? E.g., via getting aligned people in positions of power, via lobbying, via more public advocacy.
But this isn't my area; I could just be wrong. Also I imagine this is probably just a superficial phrasing thing, not an important disagreement.
Regarding the 'current approach for early warning of novel pathogens is severely lacking..'
My uneducated series of questions and thoughts are (in random order):
Founded in 2017, Phage Directory’s mission is to help unlock the untapped potential of phages for phage therapy and biocontrol by empowering people to access, use and build upon the world’s phage knowledge.
Also, I think a while back I shared a series of ideas relating to bio-security. I forget if this was way back when singularityhub used to have a forums section. I have a copy of some of my comments. This was 7 + years ago. But I think the gist is still the same. Better sensors, ability to compute information a lot more efficiently and possibly with a whole lot less energy expended. https://en.wikipedia.org/wiki/Reverse_computation
@Ethan Alley
My team and I at AmorSui have been working on bettering PPE since 2018. Our team has materials science, product design, and commercialization expertise to execute on the ideas you posted around Super PPE.
Would love to chat more to expand on the projects as we are planning on submitting an application to FTX Future Fund. Feel free to message me on LinkedIn.
-Beau
Perhaps this is naive, but would not the few, remaining uncontacted peoples provide some degree of resilience to pandemics? That is, unless for some reason during a catastrophic pandemic infected people manage to "break into" their communities? https://en.wikipedia.org/wiki/Uncontacted_peoples Without having thought deeply about this, perhaps another way to achieve biosecurity would be to support the protection of these communities and especially put in place something that increases the chance of them staying isolated in case of an outbreak (e.g. being able to give them access to test for and treat contaminated water).
Andrew Snyder-Beattie and Ethan Alley
This is a list of longtermist biosecurity projects. We think most of them could reduce catastrophic biorisk by more than 1% or so on the current margin (in relative[1] terms). While we are confident there is important work to be done within each of these areas, our confidence in specific pathways varies widely and the particulars of each idea have not been investigated thoroughly.
Still, we see these areas as critical parts of biosecurity infrastructure, and would like to see progress building them out. If you’d like to be kept up to date about opportunities to get involved, please fill out this Google Form.
Early detection of a biothreat increases the amount of time we have to respond (e.g. designing tailored countermeasures, using protective equipment, heading to bunkers, etc). The current approach for early warning of novel pathogens is severely lacking—it typically relies on a particularly astute doctor realizing that something is strange combined with negative tests for everything else. Existing systems are also almost exclusively focused on known pathogens, and we could do a lot better using pathogen-agnostic systems that can pick up unknown pathogens.
One concrete goal would be something simple where a small team of people collects samples from volunteer travelers around the world and then does a full metagenomic scan for anything that could be dangerous.[2] Even collecting and analyzing only 100 random samples per day could make a big difference in some scenarios, since that would mean we would still expect to catch things before they infect too large a fraction of the global population. We think that with the right team, this could be done with close-to-existing technology for less than $50 million per year.[3]
There are a handful of bottlenecks and a number of ways to decompose this problem. To get started on subproblems, one of us (Ethan) is working on a list of suggestions, which we will backlink here.
Most personal protective equipment (PPE) is not good enough. Things like masks and suits require training to fit properly, lack reusability, and are generally designed for routine uses rather than for the most extreme events. The small minority of PPE that is designed for extreme use cases (e.g. BSL4 suits or military-grade PPE) is bulky, highly restrictive, and insufficiently abundant—not the kind of thing you could easily put millions of healthcare/pharma/essential workers into if needed. It seems plausible that with good materials science and product design we could come up with next-generation PPE that is simultaneously highly effective in extreme cases, easy to use, reliable over long periods of time, and cheap/abundant.
One concrete commercial goal would be to produce a suit (and accompanying system) that is designed for severely immunocompromised people to lead relatively normal lives, at a cost low enough to convince the US government to acquire 100 million units for the Strategic National Stockpile.[4] Another goal would be for the suit to simultaneously meet military-grade specifications, e.g. protecting against a direct hit of anthrax.
PPE has the advantage of being truly ‘pathogen-agnostic’—we can stockpile it in advance of knowing what the threat is, in contrast to vaccines or many medical countermeasures. It is also ‘defensively stable’ in that physical barriers can’t be easily bypassed using pathogen engineering techniques (whereas many medical countermeasures might be defeated with some creative tinkering). See Carl Shulman’s post here for more on this.
To get started on subproblems within PPE, one of us (Ethan) will publish a PPE deeper dive at some point in the future (backlink forthcoming).
Medical countermeasures (e.g. vaccines, antivirals, monoclonal antibodies) for use against catastrophic biorisks have a number of existing drawbacks. In most cases they are tailored to existing pathogens (e.g. smallpox vaccines) and wouldn’t help against a novel threat. Many countermeasures are also not robust against deliberate engineering (e.g. antibiotics are broad-spectrum but can be overcome).
We think there could eventually be opportunities for radically improved medical countermeasures against GCBR-class threats, either by 1) producing targeted countermeasures against particularly concerning threats (or broad-spectrum countermeasures against a class of threats), or by 2) creating rapid response platforms that are reliable even against deliberate adversaries.
However, we are not yet ready to recommend medical countermeasures as a general focus area for large scale projects, in part because many projects in this space have inadvertent downside risks (for example, platforms that use viral vectors may accelerate viral engineering technology). If you feel excited about working in this area, fill out the Google Form (here) and we might be able to provide you some more tailored advice.
Right now, the biological weapons convention (BWC)—the international treaty that bans biological weapons—is staffed by just four people and lacks any form of verification. We think there is more scope for creative ways of strengthening the treaty (e.g. whistleblowing prizes), or creating new bilateral agreements and avoiding bureaucratic gridlock. Moreover, a team of people scouring open sources (i.e. publication records, job specs, equipment supply chains) could potentially make it difficult for a lab to get away with doing bad research, and thereby strengthen the treaty.
Sterilization techniques that rely on physical principles (e.g. ionizing radiation) or broadly antiseptic properties (e.g., hydrogen peroxide, bleach) rather than molecular details (e.g. gram-negative antibiotics) have the advantage of being broadly applicable, difficult to engineer around, and having little dual-use downside potential.
Existing technologies for physical sterilization (e.g. UV light, materials science for antimicrobial surfaces, etc.) have different limitations in terms of costs, convenience, and practicality, and we think this is an underexplored area for prevention and countermeasure development. We have a lot of remaining uncertainties in this area but think the value of investigating it is high.
Existing bunkers provide a fair amount of protection, but we think there could be room for specially designed refuges that safeguard against catastrophic pandemics (e.g. cycling teams of people in and out with extensive pathogen agnostic testing, adding a ‘civilization-reboot package’, and possibly even having the capability to develop and deploy biological countermeasures from the protected space). This way, some portion of the human population is always effectively in preemptive quarantine.
Another way of framing this: lots of people think we’d substantially reduce biorisk if we had a self-sustaining settlement on Mars (and we basically agree). If that’s the case, it would be a lot cheaper to put the exact same infrastructure on Earth, and it buys almost the same amount of protection.
One next step on this would be to build an org which specializes in the operations, logistics and contractor relationships needed to actually build a refuge with the necessary amenities (based on a shallow investigation, one of us, ASB, ballparked outsourcing at around $100-300M / bunker but did not have logistics expertise or time to dig deeper). We have some more ideas in the works we’ll backlink here later, but please fill out the form if you’re interested in the meantime.
A few things we want to highlight:
Despite how promising and scalable we think some biosecurity interventions are, we don’t necessarily think that biosecurity should grow to be a substantially larger fraction of longtermist effort than it is currently. From a purely longtermist perspective we think that AI might be between 10-100x more important than biosecurity, even if solving biosecurity might be more tractable than solving AI (possibly by a large factor). Biosecurity is also attractive as a cause area for non-longtermist reasons, given the importance of preventing lesser catastrophes that fall short of truly civilization-ending but are still horrific (e.g., 10-100x COVID)—we thus think it could be even more relatively appealing for those more focused on impacts on the current generation(s).
Again, please fill in this coordination form to stay informed of developments and opportunities.
We thank Chris Bakerlee, Jamie Balsillie, Kevin Esvelt, Kyle Fish, Cate Hall, Holden Karnofsky, Grigory Khimulya, Mike Levine, and Carl Shulman for feedback on this post.
This work is licensed under a Creative Commons Attribution 4.0 International License.
E.g. if biorisk was 1% in the next century, each of these interventions would cut the absolute risk of catastrophe by at least 0.01% ↩︎
This version of a ‘sentinel system’ is going to be neglected by traditional public health authorities and governments because they won’t be searching for engineered threats designed to elude pathogen-specific detection tools. ↩︎
Another discussion of this idea, a ‘nucleic acid observatory,’ can be found here. ↩︎
One possible downside risk is that adversarial countries might interpret such a huge bulk purchase of PPE as being evidence or preparation for strategic biological warfare, leading to an arms race dynamic. The company should therefore be cautious about how it messages and should also liberally sell this equipment everywhere in the world to signal defensive intent. ↩︎
Really happy to see this, this looks great.
This is outside the scope of this document, but I'm a bit curious how useful it would have been to have such a list 3-5 years ago, and why it took so long. Previously I heard something like, "biosecurity is filled with info-hazards, so we can't have many people in it yet."
Anyway, it makes a lot of sense to me that we have pretty safe intervention options after all, and I'm happy to see lists being created and acted upon.
The authors will have a more-informed answer, but my understanding is that part of the answer is "some 'disentanglement' work needed to be done w.r.t. biosecurity for x-risk reduction (as opposed to biosecurity for lower-stakes scenarios)."
I mention this so that I can bemoan the fact that I think we don't have a similar list of large-scale, clearly-net-positive projects for the purpose of AI x-risk reduction, in part because (I think) the AI situation is more confusing and requires more and harder disentanglement work (some notes on this here and here). The Open Phil "worldview investigations" team (among others) is working on such disentanglement research for AI x-risk reduction and I would like to see more people tackle this strategic clarity bottleneck, ideally in close communication with folks who have experience with relatively deep, thorough investigations of this type (a la Bio Anchors and other Open Phil worldview investigation reports) and in close communication with folks who will use greater strategic clarity to take large actions.
I have only been involved in biosecurity for 1.5 years, but the focus on purely defensive projects (sterilization, refuges, some sequencing tech) feels relatively recent. It's a lot less risky to openly talk about those than about technologies like antivirals or vaccines.
I'm happy to see this shift, as concrete lists like this will likely motivate more people to enter the space.
More than infohazards, we were still building capacity and understanding of the area.
But many of these were highlighted in earlier work, including decade of reports from Center for Health Security, etc. (Not to mention my paper with Dave Denkenberger.)
thanks for the kind words! I agree that we didn't have much good stuff for ppl to do 4 yrs ago when i started in bio but don't feel like my model matches yours regarding why.
But I'm also wanting to confirm I've understood what you are looking for before I ramble.
How much would you agree with this description of what I could imagine filling in from what you said re 'why it took so long':
"well I looked at this list of projects, and it didn't seem all that non-obvious to me, and so the default explanation of 'it just took a long time to work out these projects' doesn't seem to answer the question"
(TBC, I think this would be a very reasonable read of the piece, and I'm not interpreting your question to be critical tho also obviously fine if it is hahah)
That sounds like much of it. To be clear, it's not that the list is obvious, but more that it seems fairly obvious that a similar list was possible. It seemed pretty clear to me a few years ago that there must be some reasonable lists of non-info-hazard countermeasures that we could work on, for general-purpose bio safety. I didn't have these particular measures in mind, but figured that roughly similar ones would be viable.
Another part of my view is,
"Could we have hired a few people to work full-time coming up with a list about this good, a few years earlier?"
I know a few people who were discouraged from working in the field earlier on because their was neither the list, nor the go-ahead to try to make a list.
I don't think any of the info hazards are mentioned here, but you're right that good lists like this are a long time coming. I haven't heard that biosec folks actively didn't want people in the field though-- would be interested in who said that.
FWIW, I know of a case from just last month where an EA biosecurity person I respect indicated that they or various people they knew had substantial concerns about the possibility of other researchers (who are known to be EA-aligned and are respected by various longtermist stakeholders) entering the space, due to infohazard concerns.
(I'm not saying I think these people should've been concerned or shouldn't have been. I'm also not saying these people would have confidently overall opposed these researchers entering the space. I'm just registering a data point.)
I am surprised, and feel like I need more context. "This space" is probably too vague. I'm definitely opposed to even well-aligned people spending time thinking up new biothreats. But that's very different than working on specific risk mitigation projects.
By "this space", I meant the longtermist biosecurity/biorisk space. As far as I'm aware, the concern was along the lines of "These new people might not be sufficiently cautious about infohazards, so them thinking more about this area in general could be bad", rather than it being tailored to specific projects/areas/focuses the new people might have (and in particular, it wasn't because the people proposed thinking up new biothreats).
(But I acknowledge that this remains vague, and also this is essentially second-hand info, so people probably shouldn't update strongly in light of it.)
I would agree that getting people who aren't cautious about things like infohazards is a much more mixed blessing if we're talking about biorisk generally, and I'd want to hear details about what they were doing, and why there were concerns. (I can think of several people whos contribution is net-negative because most of what they do is at best useless, and they create work for others to respond to.)
But as I said, the pitch here from ASB and Ethan was far more narrow, and mostly avoids those concerns.
It seems reasonable to me to be vigilant of sharing infohazards with new researchers in the field. Still, I am wondering if it might actually be worse to leave new researchers in the dark without teaching them how to recognize and contain those infohazards, especially when some are accessible on the internet. Is this a legitimate concern?