If we were spending $10M a year instead of $1.5M on sequencing, how much less than 0.2% of people would have to be infected before an alert was raised?

It's pretty close to linear: do 10x more sequencing and it goes from 0.2% to 0.02%. You can play with our simulator here: https://data.securebio.org/simulator/

How should I feel about 0.2%? Where is 0.2% on the value spectrum from no alert system and an alert system that triggered on a single infection?

That's an important question that I don't have the answer to, sorry!

How many people's worth of wastewater can be tested with $1.5M of sequencing?

This isn't a question of limits, but of diminishing returns to sampling from additional sewersheds. Which also depends a lot on how different the sewersheds are from each other.

Here you go: Detecting Genetically Engineered Viruses With Metagenomic Sequencing

But this was already something I was going to put on the Forum ;)

Very little biorisk content here, perhaps because of info-hazards.

When I write biorisk-related things publicly I'm usually pretty unsure of whether the Forum is a good place for them. Not because of info-hazards, since that would gate things at an earlier stage, but because they feel like they're of interest to too small a fraction of people. For example, I could plausibly have posted Quick Thoughts on Our First Sampling Run or some of my other posts from https://data.securebio.org/jefftk-notebook/ here, but that felt a bit noisy?

It also doesn't help that detailed technical content gets much less attention than meta or community content.  For example, three days ago I wrote a comment on @Conrad K.'s thoughtful Three Reasons Early Detection Interventions Are Not Obviously Cost-Effective, and while I feel like it's a solid contribution only four people have voted on it.  On the other hand, if you look over my recent post history at my comments on Manifest, far less objectively important comments have ~10x the karma. Similarly the top level post was sitting at +41 until Mike bumped it last week, which wasn't even high enough that (before I changed my personal settings to boost biosecurity-tagged posts) I saw it when it came out.  I see why this happens--there are a lot more people with the background to engage on a community topic or even a general "good news" post--but it still doesn't make me as excited to contribute on technical things here.

It seems to me that being considerate of others is still valuable even (especially!) if you're trying to convince them that they're making a horrendous moral error. Almost all attempts to change someone's mind in this way don't work, and failed attempts often contribute to memetic immunity. History is full of disagreeable-but-morally-correct people being tragically unpersuasive.

I work on the Nucleic Acid Observatory at SecureBio, but in this comment I'm speaking only for myself.

Thanks for writing this, and thanks @mike_mclaren for sending it to me! I wish I'd happened across it when you'd posted it [1] and could have given more timely feedback!

At a high level I think this is very reasonable, especially sections #1 and #3: it's very important that you can turn "here is a collection of highly suspect sequencing reads" into a well-calibrated and trusted "here is how concerned we should be", and it's also very important that you can then turn that into "and here's what we will do".  I've seen scattered work on both of these, but I think these are key areas for additional work.

On section #2, which is closer to what I've been thinking about, there were some places where I wanted to push back or add additional information.  A few point-by-point replies:

I even found that disease surveillance is amongst the most well-funded issue priorities within biosecurity. ...However, insofar as I used to think work on early detection is both one of the safest bets for GCBR mitigation and also not very neglected, I no longer think either of these to be true.

While there is a lot of spending on surveillance, this is overwhelmingly allocated to tracking known threats: how prevalent are SARS-CoV-2, Influenza, RSV, etc. I'm not aware of anyone who has deployed a system capable of alerting on a novel pathogen. While (in my work at the NAO) I'd like to change this, I do still think it's quite neglected for now.

Wastewater surveillance data has the disadvantage of not necessarily being connected to a patient or the identifiable spread of a pathogen.

This is definitely a real drawback.  On the other hand, once your wastewater-based initial-detection system flags a pathogen you have what you need to create a cheap PCR-based diagnostic, which you can then deploy widely to identify infected individuals (to see how they're doing, if they need treatment, and understand the effect the pathogen is having in their bodies, and also to limit spread). This could let you identify many more infected individuals than if you plan was for your initial-detection system to also double as your identification-of-individuals system.  But the need for follow-up work ties into the rest of your post, which is that initial detection is only one piece, and if you don't have follow-ups ready to go you'll lose many valuable doubling periods fumbling the next steps.

For various surveillance modalities, lowering time-to-confirmation is an important part of lowering time-to-detection.

Strong agree!

Regularly screening 1% of the population every day is a lot. In the UK, this would be ~680,000 people per day, just over half the 1.3m people the NHS sees every day. As a very loose estimate, we can anchor on the costing provided in the proposed Threat Net architecture. Threat Net has the all-in variable cost per test at roughly $1500[10] in the US. Assuming 1 test = 1 sequencing run (e.g. so it also serves a diagnostic purpose), it would cost upwards of $372bn per year to test this many people[11]. Even assuming 10 samples can be multiplexed, £37.2bn would be close to 20% of the UK’s healthcare budget. Waiting to multiplex many more samples than this would contravene the purpose of early detection.

While I agree that daily screening of 1% of the UK's population would be very expensive, I think the main cost (unless you're doing wastewater) is in collecting the samples.  That's just a massive endeavor! If you get sampling down to $1 person it's still $250M/y (68M people × 1% × 365) in sample collection alone.  This range might be possible if you somehow just made it part of the culture (ex: every school child provides a sample every morning) but would be a huge lift.

But I think your estimates for the cost are otherwise too high.  The main thing is that if you do pooled testing you can cost-effectively sample hundreds of thousands of people at once.  This isn't "multiplexing" because you're not adding barcodes (or something else) that would allow you to map sequencing reads back to specific sampled people. For example, say you're doing pooled nasal swabs, and you're sampling a large enough number of people that you'll likely get some high viral load folks.  Then the average contribution of a sick person is a relative abundance of maybe 2% (rough analysis; @slg and @Will Bradshaw should have a much more thorough treatment of this out any day now).  If you run a NovaSeq X 25B (which gets you 25B sequencing read pairs) on a massively pooled sample and 1:100,000 people are currently shedding, you'll get about 5k (2% ÷ 10,000 × 25B) read pairs matching the virus, which is likely (again, hoping to get something out publicly about this soon!) enough to flag it. The list price for the flow cell for a 25B sequencing run is $16k, which is perhaps half the all-in cost.  Which means if you did daily NovaSeq X 25B runs you'd be spending $10-$20M/y ($16k × 2 × 365) on the sequencing.  

This is all very ballpark, but mostly is to say that if you're talking about directly sampling that many people the main cost is in the collection.

This is quite different if you're using wastewater to avoid needing to sample that many people, but then your relative abundances are far lower and you need much deeper sequencing (expensive!) to get anywhere close to detection at 1:100,000.

populations that exhibit much higher levels of viral load in the early stages of an infection (e.g. due to weaker immune systems)

I was briefly quite excited about this direction, but a major issue here is that these populations are, because of their vulnerability, somewhat epidemiologically isolated: they're probably not close to the first people getting sick in anywhere.

Ideally, we should not be content with a 5% chance of a detection system failing us for a GCBR.

That seems too strong, at least in theory.  We should evaluate the marginal detection effort in terms of how much it reduces risk for the cost.  For example, a detection system that was very cheap to deploy and only flagged a pandemic 30% of the time could be well worth deploying even though it most likely doesn't help.  And perhaps scaling it up to 90% probability of detection would remain cost effective, but getting it up to 99% would be massively more expensive for diminishing returns and so doesn't work out. On the other hand, political considerations might come into play: a system that on average doesn't work could be a hard sell even if it's relatively cheap.

My first takeaway is a prima facie scepticism about the value of early detection for preventing GCBRs in particular compared to preventative measures - with the exception of stealthier pandemics.

I also don't expect work focused on initial-detection to help much outside of stealth pandemics.  I see my work at the NAO as almost entirely valuable in proportion to the likelihood that someone creates (or perhaps would create, if not deterred by the likelihood of it being flagged) a stealth pandemic.

That said, there is one other situation I've seen bounced around where I think this kind of monitoring could be valuable outside of stealth scenarios: when we know there's something happening in a region inside a country we have very poor relations with.  In that case if we had a system of ongoing monitoring we could pivot it to populations enriched for contact with that region and (ideally) ramp it up.

[1] A few weeks ago I happened to go through and changed my settings to give a big boost to any biosecurity-tagged posts, so I think if this came through again I'd see it.

i'm a little confused about the goals of a lot of the folks who're commenting. like, their (your?) marginal 20 minutes would be WAY more effective by... idk, hopping on a call with me or something?

To the extent that people are trying to influence future Manifest decisions or your views in particular, I agree that 1:1 private discussion would often be better. But I read a lot of the discussion as people trying to hash out what EA community (broadly construed) norms should be in this sort of situation, and that seems to me like it's reasonably well suited for public discussion?

You're right that I don't have to talk about everything that I find. To take an uncontroversial example, if in my day job I find an easy way to make a bioweapon, I'm not going to blog how to do that.

But if you're not going to talk about it if you conclude X, are you also not going to talk about it if you conclude not-X? If not then you're making it much harder for other people to figure out what is true (more).

The asymmetry that @Ben Millwood points to below is important, but it goes further. Imagine a hundred well-intentioned people look into whether there are dragons. They look in different places, make different errors, and there are a lot of things that could be confused for dragons or things dragons could be confused for, so this is a noisy process. Unless the evidence is overwhelming in one direction or another, some will come to believe that there are dragons, while others will believe that there are not.

While humanity is not perfect at uncovering the truth in confusing situations, our approach that best approaches the truth is for people to report back what they've found, and have open discussion of the evidence. Perhaps some evidence A finds is very convincing to them, but then B shows how they've been misinterpreting it. Except this doesn't work on taboo topics:

• Many sensible people have (what I interpret as) @NickLaing's perspective, and people with that perspective will only participate in the public evidence reconciliation process if they failed to find dragons. I don't know, for example, whether this is your perspective.
  • You wrote essentially the opposite ("Those who perceive 'dragons on the map' will often feel their integrity is at stake unless they speak up. Those who didn't find any will lose interest and won't feel their integrity is at stake, so they won't speak up.") and I agree some people will think this way, but I think this is many fewer people than are willing to publicly argue for generally-accepted-as-good positions but not generally-accepted-as-evil ones.
• Many people really do or don't want dragons to exist, and so will argue for/against them without much real engagement with the evidence.
• Good faith participation in a serious debate on the existence of dragons risks your reputation and jeopardizes your ability to contribute in many places.

So I will continue not engaging, publicly or privately, with evidence or arguments on whether there are dragons.

Does advocating the anti-Bob position in any way constitute not "treat[ing] him respectfully," even if he is not in earshot?

Let's say that Alice is going to advocate for her anti-Bob position even when Bob is in the discussion. And that this is a carve out from "treat Bob respectfully".

I think you'd be a lot more successful with a hypothetical that wasn't about whether someone would follow the law and/or conference rules.

I would also expect, for example, a conference under Chatham House Rules to reject participants who believed this kind of rule did not bind them. Even if the organizers otherwise were quite committed to free expression. Organizers can and should be willing to consider expressed beliefs even without a history of acting on them.

I also think it being about "people named Bob" messes with our intuitions, since it's so silly, but ok.

Perhaps better hypothetical would be if Alice believed people named Bob were not moral patients (that had a bunch of nasty views downstream from this on what law and social norms should be) but still confirmed (and organizers trusted) that she would follow the law and treat him respectfully at the conference?