Comments
The Apart Research AIxBio Hackathon is coming up this weekend. I work on a DNA screening tool, like making lists of projects, and think some hackathon projects would be much more useful to the biosecurity ecosystem me than others. For example, I’ve received some red-teaming disclosures that were not particularly useful, because it was not clear whether the obfuscated sequences were functional, and I take false positives really seriously.[1]
Here are a few things that I[2] would be excited to see people hack on, split by the hackathon track they seem most relevant to:
DNA Screening & Synthesis Controls
Handling many sequences at once: oligo pools and split orders
People order very large numbers of short oligos from synthesis providers; a single order might contain 150,000 80bp oligos. What are these oligos? Could they be assembled into SOCs? Wouldn’t I like an efficient way to know!
The current guidance says to use a “short sequence alignment software”. To wit:
- UK screening guidance: encourages screening all sequences ordered by an individual User, using a short sequence alignment software package. If there is alignment between any constitutes of a User’s order with a SOC, or if the sequences could be constructed to form a SOC, providers should undertake follow-up screening
- EU Biotech Act Annex I: covers any sequence which has the potential to be assembled into a sequence that is at least 200 nucleotides in length and meets point (a) or (b) if combined with other synthetic nucleic acids
- US Screening Framework: apply screening methods that detect the potential for shorter nucleotide sequences to be assembled into SOCs when multiple synthetic nucleic acids are ordered by the same customer in a bulk order or in multiple orders over time
Surely someone who is smart about information theory and string handling could do better than “use an alignment package”. I think there are likely to be possible efficiency gains in representing the orders (e.g. if you have a 60,000-oligo mutagenesis library, I feel like you could represent all the information in it in a smaller number of sequences to screen). Additionally, I don’t know the best approach for recognizing the possibility that a set of sequences could be assembled (see note on assembly signatures below).
Detecting useful signatures within orders
You could build out algorithms + test datasets for detecting decision-relevant signatures within DNA orders, such as:
Assembly signatures
I don’t have a convenient reference set of “all of the molecular signatures that should clue me into someone following a standard assembly protocol”, and it would be nice to be able to tell my users about this / prioritize those orders for assembly-and-review. This would also potentially enable more logical assembly than “use an alignment package”.
DNA that is chemistry, rather than biology
I would love to see a quick comparison of approaches for recognizing when DNA is abiological in nature (DNA origami, DNA data storage, etc) as well as some analysis of how robust those approaches are. I listed some possible approaches in this doc: Distinguishing biological and abiological sequences (detecting AI-generated sequences) [shared].
Chimera / engineering detection
This is relevant for the hackathon track on surveillance and detection as well. As a few starting points, I’d check SecureBio Detection’s algorithms: Detecting Genetically Engineered Viruses With Metagenomic Sequencing – SecureBio as well as (lower confidence) Synsor: a tool for alignment-free detection of engineered DNA sequences - PMC, Tracing engineered biothreats with AI forensics: Five steps to improve attribution - Bulletin of the Atomic Scientists, and this writeup on FELIX.
Expanding SOC databases with predicted functional variants
This is way too big for a weekend, but I know some of y’all want to make pipelines for AI-generated SOCs (in fairness: me too). Right now, I’m most excited about a database expansion approach, since it could make screening more robust without requiring functional prediction to be run on each ordered sequence. Many screening tools are already doing this in some form and several have published their approaches for including likely-functional variants in their sequence search[3].
Note that I’ve tried to frame this in a fairly narrow way; I suspect designing a pipeline for obfuscating SOCs, or designing totally novel SOCs, would be a poor use of your hackathon time (you could easily spend the whole weekend on that pipeline, creating more risk than you reduce). If you do design an approach for obfuscating SOCs, please disclose to the SBRC before public disclosure so we can ensure the major screening tools are able to detect those sequences.
If you’re interested in this, I recommend two recent preprints:
- Beyond Sequence Similarity: Toward Function-Based Screening of Nucleic Acid Synthesis (disclosure: self-citation) which cites various approaches, including this SecureDNA whitepaper (see github repo) and this RTX BBN whitepaper
- ADAPT: a Programme for the Advanced Detection of AI-enabled Pathogenic Threats, which outlines a possible successor program to FunGCAT
If you wanted to start on this, you could work on functional variants for some of the classes of protein mentioned in Beyond Sequence Similarity:
Protein cytotoxins represent the clearest starting point: the relationship between structure and function is comparatively well-understood, decades of toxicology research provide structure-activity relationships and characterized variants, the mechanistic space is bounded, and detection aligns with current regulation of controlled toxins. Viral entry proteins, particularly receptor-binding proteins for pandemic-capable viruses, would be a natural next step.
Better in-silico scores than TM and ΔpLDDT
One sub-project I’d be interested in would be some automated scoring of whether AI-generated variants are “likely functional”. In the Paraphrase project, we used a loose cutoff where we deemed sequences with TM Score > 0.5 and ΔpLDDT > -10.0 as “more-probably-functional”. This is extremely basic and verifiably kind of wrong. There are surely more sophisticated scoring metrics for this (I have only skimmed this review, but maybe it has some?) and it would be great to be able to score variants before incorporating them into standard benchmarking and test sets.
Cross-Track: customer screening mini tools
Customer screening is relevant to DNA synthesis screening (my life), but also frontier AI trusted access programs, managed access to biological data, and many other dual-use life sciences products. There are now several teams actively developing customer screening tools (e.g. Aclid, Cliver, TwentyTwo) and IBBIS and EBRC are currently working on standards for customer screening in DNA synthesis. See also recent publications on customer screening from Cliver, IBBIS, and NTI.
I wouldn’t recommend you scrap together an end-to-end KYC tool at a hackathon, but rather that you try to automate a few steps of customer verification really well. Here are a few that seem well-suited to hacking:
Affiliation verification
I put a few ideas on affiliation verification here. An API that gave me a rapid confidence rating for whether a name + affiliation pair are actually associated would be nice.
Address verification
There are tools for this but I don’t know if they’re any good ― the thing I really want is “be sure that this address is in fact associated with this named institution”. Alejo Acelas has already generated some good thoughts on this.
National and local government approval verification
The idea (described in a recent IBBIS whitepaper) would be to create a guide to leveraging existing approvals to demonstrate customer legitimacy. Many customers already have to get government or institutional approvals to work with pathogens or recombinant DNA. This would build on policy / licensing information from the Synthesis Map to create a guide to when synthesis providers should be able to expect customers to have a license, committee approval, etc. This project is inspired in part by SecureDNA Exemption Certificates, but also by the existence of licensing for recombinant DNA work in many regions. (NB: IBBIS is probably going to work on this later this year, but ideas are cheap and I’d be happy if you beat us to it.)
Benchtop Synthesis
On-device record-keeping
How could you implement record-keeping for benchtop devices, for the purposes of later auditing? Is it possible to compress data in a way that enables (secure) on-device record-keeping? Think “aircraft black box”, but for benchtop synthesizers. See Baker & Church in “Protein design meets biosecurity” and Jonas Sandbrink on “record-keeping for strong attribution”.
On-device split order detection
Once you have record-keeping, maybe you can have on-device detection of split orders intended to be assembled into SOCs? A reasonable conclusion from this project might be “no you can’t, if you want this you need to send sequences to some shared online database”.
Tamper-evident or tamper-proof approaches for benchtop devices
Tampering is one of the technical mitigations in the IFP report on Securing Benchtop DNA Synthesizers. What are the most tamper-proof laboratory devices right now? How viable / expensive is this?
Examining the reagent supply chain: any hope of KYC?
“Develop a standardized benchtop and reagent customer screening process” is one of the recommendations in the IFP report on Securing Benchtop DNA Synthesizers. Reagent tracking is also highlighted in the UK screening guidance and in a 2018 DEFCON talk by Meow Ludo Meow Meow. It would be useful to know whether all current benchtop devices work with similar/generic dNTPs, how many different devices have proprietary reagents, and whether enzymatic or chip-based synthesis changes anything.
On-device authentication and authorization
It would be good to have some experiments on secure authentication and authorization systems for customers who have been approved to work with SOCs. Spoofing and Elevation of Privilege are among the technical mitigations in the IFP report on Securing Benchtop DNA Synthesizers. The SecureDNA Exemption Certificates are one approach, involving hardware keys and approvals by biosafety officers.
AI Biosecurity Tools
Biosecurity Statements Repository++
Last year Max Langenkamp and I wrote up Five Tractable Biosecurity Projects You Could Start Tomorrow. In response Shiying He and P.T. Nhean created a Biosecurity Statements Repository which they described thus:
Our repository collects around 10–20 examples of biosecurity statements and practices from biological design tools and AI models. We grouped them based on how different developers are approaching biosecurity and dual-use risk. Users would be able to get a quick glimpse at how the current landscape looks. You can think of it as a map of how the field is (and isn’t yet) talking about biosecurity risks and safety.
I’m so happy they did this, but have realised I feel odd linking an EA Forum Post for an audience generally distant from the community (BDT developers). I think a speedy vibecode to turn this into a set of tables / graphs / etc on a website, rather than a Google sheet, would be quite valuable ― something lightweight like virologytest.ai.
This field also moves extremely fast, and so many recent dual-use model statements from our Evos, Alphas, Codexes, etc. are not present. Please make this resource so I can link to it! (Note that Shiying and PT have given their blessing for others to pick up this project; please reference them as the curators of the original dataset!)
Screening-in-the-tool-use-loop
I worry about a scenario where it’s hard to tell that an LLM is being used in a potentially risky way, because all the signals are in the data operated on by tool calls, rather than any natural language. Can we recognize when tool calls are operating on biological sequence data and trigger a sequence screen call? Does it need to be for every file, or just occasional random anomaly checks?
I’m on vacation during the AIxBio Hackathon this weekend, but if you end up working on one of those projects, either during that weekend or later, and want to discuss them with me, feel free to reach out to my first name @ibbis.bio with the subject “AI x Bio Hackathon Project”, describing what you’ve worked on, and we can find a time for a quick call.
The header image for this post is a collage of Staph Aureus plates from HansN on Wikimedia Commons and a photo from a 1984 “computer festival in Amsterdam” credited to the National Archives of the Netherlands.
- ^
Not purely because it harms adoption by annoying providers (though that’s a factor) but also because it creates alert fatigue that could cause the humans in the loop to miss important flags.
- ^
Most of these ideas are not original, and a few grew directly out of pre-hackathon brainstorming with my IBBIS colleague Lucas Boldrini.
- ^
You can see a sketch of the sort of screening pipeline we’re working to implement in Commec this year in this doc. See also this SecureDNA-related repo and the SeqScreen paper.
