Agreed that the 3-log reduction is appropriate for cases where the “baseline” dose is pretty low. Our team is still uncertain about what protection level might be needed, but has some high-level reasons to expect that background environment-to-human (E2H) aerosol pathogen concentrations are likely low in many environments.
One reason is that airborne microbial concentrations and species composition vary a lot across environments. Existing microbial ecosystems also tend to be quite diverse, so even the most common bacterial species often make up only a small share of total bacteria. Likewise, an E2H pathogen, such as a mirror bacterium, probably wouldn’t become highly concentrated across most natural ecosystems. While mirror bacteria might not face the same predation pressures as many other bacteria, they would still have to compete for resources and wouldn’t necessarily occupy a large share of many ecological niches. So even though outdoor air can contain thousands to millions of bacteria per cubic meter, mirror bacteria probably wouldn’t be present at high concentrations in many outdoor environments. To the extent concentrations vary substantially by location, there might also be places with much lower exposure risk.
There are also reasons to expect that the infectious dose wouldn’t necessarily be one bacterium. Even if human immune systems did not respond to mirror bacteria, our respiratory tracts have physical defenses, such as blocking or clearing inhaled particles, and bacteria do not automatically colonize every environment they land in. The BPP team is very unsure what an infectious dose might be, and it could be low—for infections like Q fever and TB, I think the infectious dose is quite low—but it is not guaranteed to be low for E2H pathogens.
Last, the 3-log reduction is what our team currently thinks could buy humanity enough run room to find better solutions, such as building better protections or developing and mass-producing medical countermeasures (MCMs). So the 3-log protection doesn't need to provide perfect protection for someone’s whole life, but might just keep people alive for a year or a few years. Also, in the early stages of an E2H pathogen release, while humanity is increasing resilience through better protection options or MCMs, concentrations might still be lower as the pathogen spreads, so the threshold for useful protection might be lower.
Agreed that measuring inside negative-pressure respirators is harder, but it might be more feasible to measure particle concentrations inside improvised positive-pressure respirators.
One thing that might need additional R&D on top of existing low-cost particle sensors: making sure that they can meaningfully measure very-low concentrations, e.g. air that’s 1000x cleaner than usual (which might be close to the noise floor of these sensors).
Hi Vaipan, I’ll take your question about the ratio of hires between AI safety and biosecurity. In short, no, it wouldn’t be correct to conclude that biosecurity is more talent constrained than AI safety. The number of roles is rather a reflection of our teams’ respective needs at the given moment.
And on the “more diverse consideration about GCR” question, note that my team is advertising for a contractor who will look into risks that lie outside biosecurity and AI safety, including nuclear weapons risks. Note though that I expect AI safety and biosecurity to remain more highly prioritized going forward.
As highlighted in the job descriptions, the answer for what we’re looking for in both skills and experiences varies from role to role (i.e., operations ≠ grantmaking ≠ research). When we evaluate candidates, though, we usually are less asking “What degree do they have? How many years have they worked in this field?” and more asking “What have they accomplished so far (relative to their career stage)? Do they have the skills we’re looking for? Will they be a good fit with our team and help meet an ongoing need? Do we have reason to expect them to embody Open Philanthropy’s operating values from day one?”
Yeah, I can relate, these sorts of situations can be tough.
I work on the biosecurity & pandemic preparedness team at Open Philanthropy. In the realm of biosecurity at least, I'm happy to be a resource for helping troubleshoot these sorts of issues, including both general questions and more specific concerns. The best way to contact me, anonymously or non-anonymously, is through this short form.
Importantly, if you're reaching out, please do not include potentially sensitive details of info hazards in form submissions – if necessary, we can arrange more secure means of follow-up communication, anonymous or otherwise (e.g., a phone call).
To build on Linch's response here: I work on the biosecurity & pandemic preparedness team at Open Philanthropy. Info hazard disclosure questions are often gnarly. I'm very happy to help troubleshoot these sorts of issues, including both general questions and more specific concerns. The best way to contact me, anonymously or non-anonymously, is through this short form. (Alternatively, you could reach my colleague Andrew Snyder-Beattie here.) Importantly, if you're reaching out, please do not include potentially sensitive details of info hazards in form submissions – if necessary, we can arrange more secure means of follow-up communication, anonymous or otherwise (e.g., a phone call).
To your first point, I've modified the title to signal that this is a linkpost.
To your second point, this list is more concise in that it's only ~35 items right now, which compares to ~60 in Greg's list and ~90 in Tessa's. It may be more friendly to newcomers in the sense that it may just be less overwhelming due to its brevity, and it includes fewer dense governmental reports and academic papers.
But overall I think different lists will work for different people, and for whatever reason when I made this list this is the presentation that struck me as aesthetically fitting the bill. Other people may disagree about which format is more useful, and my guess is that ultimately they're just complementary.
I share in your hope that the attention to biorisks brought about by SARS-CoV-2 will make future generations safer and prevent even more horrible catastrophes from coming to pass.
However, I strongly disagree with your post, and I think you would have done well to heavily caveat the conclusion that this pandemic and other "endurable" disasters "may be overwhelmingly net-positive in expectation."
Principally, while your core claim might hold water in some utilitarian analyses under certain assumptions, it almost definitely would be greeted with unambiguous opprobrium by other ethical systems, including the "common-sense morality" espoused by most people. As you note (but only in passing), this pandemic truly is an abject "tragedy."
Given moral uncertainty, I think that, when making a claim as contentious as this one, it's extremely important to take the time to explicitly consider it by the lights of several plausible ethical standards rather than applying just a single yardstick.
I suspect this lack of due consideration of other mainstream ethical systems underlies Khorton's objection that the post "seems uncooperative with the rest of humanity."
In addition, for what it's worth, I would challenge your argument on its own terms. I'm sad to say that I'm far from convinced that the current pandemic will end up making us safer from worse catastrophes down the line. For example, it's very possible that a surge in infectious disease research will lead to a rise in the number of scientists unilaterally performing dangerous experiments with pathogens and the likelihood of consequential accidental lab releases. (For more on this, I recommend Christian Enemark's book Biosecurity Dilemmas, particularly the first few chapters.)
These are thorny issues, and I'd be more than happy to discuss all this offline if you'd like!
Agreed that the 3-log reduction is appropriate for cases where the “baseline” dose is pretty low. Our team is still uncertain about what protection level might be needed, but has some high-level reasons to expect that background environment-to-human (E2H) aerosol pathogen concentrations are likely low in many environments.
One reason is that airborne microbial concentrations and species composition vary a lot across environments. Existing microbial ecosystems also tend to be quite diverse, so even the most common bacterial species often make up only a small share of total bacteria. Likewise, an E2H pathogen, such as a mirror bacterium, probably wouldn’t become highly concentrated across most natural ecosystems. While mirror bacteria might not face the same predation pressures as many other bacteria, they would still have to compete for resources and wouldn’t necessarily occupy a large share of many ecological niches. So even though outdoor air can contain thousands to millions of bacteria per cubic meter, mirror bacteria probably wouldn’t be present at high concentrations in many outdoor environments. To the extent concentrations vary substantially by location, there might also be places with much lower exposure risk.
There are also reasons to expect that the infectious dose wouldn’t necessarily be one bacterium. Even if human immune systems did not respond to mirror bacteria, our respiratory tracts have physical defenses, such as blocking or clearing inhaled particles, and bacteria do not automatically colonize every environment they land in. The BPP team is very unsure what an infectious dose might be, and it could be low—for infections like Q fever and TB, I think the infectious dose is quite low—but it is not guaranteed to be low for E2H pathogens.
Last, the 3-log reduction is what our team currently thinks could buy humanity enough run room to find better solutions, such as building better protections or developing and mass-producing medical countermeasures (MCMs). So the 3-log protection doesn't need to provide perfect protection for someone’s whole life, but might just keep people alive for a year or a few years. Also, in the early stages of an E2H pathogen release, while humanity is increasing resilience through better protection options or MCMs, concentrations might still be lower as the pathogen spreads, so the threshold for useful protection might be lower.
Agreed that measuring inside negative-pressure respirators is harder, but it might be more feasible to measure particle concentrations inside improvised positive-pressure respirators.
One thing that might need additional R&D on top of existing low-cost particle sensors: making sure that they can meaningfully measure very-low concentrations, e.g. air that’s 1000x cleaner than usual (which might be close to the noise floor of these sensors).
Hi Vaipan, I’ll take your question about the ratio of hires between AI safety and biosecurity. In short, no, it wouldn’t be correct to conclude that biosecurity is more talent constrained than AI safety. The number of roles is rather a reflection of our teams’ respective needs at the given moment.
And on the “more diverse consideration about GCR” question, note that my team is advertising for a contractor who will look into risks that lie outside biosecurity and AI safety, including nuclear weapons risks. Note though that I expect AI safety and biosecurity to remain more highly prioritized going forward.
As highlighted in the job descriptions, the answer for what we’re looking for in both skills and experiences varies from role to role (i.e., operations ≠ grantmaking ≠ research). When we evaluate candidates, though, we usually are less asking “What degree do they have? How many years have they worked in this field?” and more asking “What have they accomplished so far (relative to their career stage)? Do they have the skills we’re looking for? Will they be a good fit with our team and help meet an ongoing need? Do we have reason to expect them to embody Open Philanthropy’s operating values from day one?”
Hi more better,
Yeah, I can relate, these sorts of situations can be tough.
I work on the biosecurity & pandemic preparedness team at Open Philanthropy. In the realm of biosecurity at least, I'm happy to be a resource for helping troubleshoot these sorts of issues, including both general questions and more specific concerns. The best way to contact me, anonymously or non-anonymously, is through this short form.
Importantly, if you're reaching out, please do not include potentially sensitive details of info hazards in form submissions – if necessary, we can arrange more secure means of follow-up communication, anonymous or otherwise (e.g., a phone call).
To build on Linch's response here:
I work on the biosecurity & pandemic preparedness team at Open Philanthropy. Info hazard disclosure questions are often gnarly. I'm very happy to help troubleshoot these sorts of issues, including both general questions and more specific concerns. The best way to contact me, anonymously or non-anonymously, is through this short form. (Alternatively, you could reach my colleague Andrew Snyder-Beattie here.) Importantly, if you're reaching out, please do not include potentially sensitive details of info hazards in form submissions – if necessary, we can arrange more secure means of follow-up communication, anonymous or otherwise (e.g., a phone call).
Thanks for the feedback, Vaidehi.
To your first point, I've modified the title to signal that this is a linkpost.
To your second point, this list is more concise in that it's only ~35 items right now, which compares to ~60 in Greg's list and ~90 in Tessa's. It may be more friendly to newcomers in the sense that it may just be less overwhelming due to its brevity, and it includes fewer dense governmental reports and academic papers.
But overall I think different lists will work for different people, and for whatever reason when I made this list this is the presentation that struck me as aesthetically fitting the bill. Other people may disagree about which format is more useful, and my guess is that ultimately they're just complementary.
I share in your hope that the attention to biorisks brought about by SARS-CoV-2 will make future generations safer and prevent even more horrible catastrophes from coming to pass.
However, I strongly disagree with your post, and I think you would have done well to heavily caveat the conclusion that this pandemic and other "endurable" disasters "may be overwhelmingly net-positive in expectation."
Principally, while your core claim might hold water in some utilitarian analyses under certain assumptions, it almost definitely would be greeted with unambiguous opprobrium by other ethical systems, including the "common-sense morality" espoused by most people. As you note (but only in passing), this pandemic truly is an abject "tragedy."
Given moral uncertainty, I think that, when making a claim as contentious as this one, it's extremely important to take the time to explicitly consider it by the lights of several plausible ethical standards rather than applying just a single yardstick.
I suspect this lack of due consideration of other mainstream ethical systems underlies Khorton's objection that the post "seems uncooperative with the rest of humanity."
In addition, for what it's worth, I would challenge your argument on its own terms. I'm sad to say that I'm far from convinced that the current pandemic will end up making us safer from worse catastrophes down the line. For example, it's very possible that a surge in infectious disease research will lead to a rise in the number of scientists unilaterally performing dangerous experiments with pathogens and the likelihood of consequential accidental lab releases. (For more on this, I recommend Christian Enemark's book Biosecurity Dilemmas, particularly the first few chapters.)
These are thorny issues, and I'd be more than happy to discuss all this offline if you'd like!