Hide table of contents

Tl;dr: Technical progress in DNA synthesis has outpaced regulatory safeguards against negligent and malevolent misuse of DNA synthesis technology. The need for intervention from the community has been on our collective radar for sometime and still coordination on the hottest issues looks underwhelming. 

Epistemic status: Low - Master's degree in epidemiology and broad expertise in public sector/health policy but <5 hours reading on this issue, synthetic biology, bioterrorism etc

The conversation on biosecurity in the EA community appears to somewhat infrequent and when it does to me it appears immature - short of leadership and expertise proportionate to the imminence, scale, neglectedness and tractability of the threat and solution, and the standard of analysis we expect in analyses and ideas proposed by EAs in the global health and development space. 

Background

The problem is that DNA synthesis procedures enable persons to render pathogens undetectable by established methods, vaccines ineffective and increase virulence or lethality [source]. In a pandemic simulation by the Johns Hopkins Center for Health Security in 2018, a terror group modeled on Aum Shinrikyo engineered a virus that killed 150 million people [source]. At the 2021 Munich Security Conference, the Nuclear Threat Initiative wargamed an exercise where a terrorist group engineered a strain of monkeypox in a country with weak oversight that left 270 million dead and more than three billion sickened [source].

Urgency

Engineered viruses are already cheap and could be even more readily available to bioterrorists or negligent entities in the immediate future:

  • DNA synthesis technology lets you “print” the sequence and create your own copy of the virus once you have the digital sequence for a biological agent. [source]. The cost of DNA sequencing dropped from $2.7 Billion to $300 in under 20 Years [source]. The cost of gene synthesis per base pair has dropped from approximately $10 a base pair to approximately $0.10 a base pair over the past 10 years [source]. The smallest viral genomes are 1000-2000 base pairs [source].
  • By one estimate [source] well short of AI Impact’s survey standards, tabletop DNA synthesis that could widely proliferate biological weapon capabilities are less than 20 years away.  

Capital

U.S. federal government funding for civilian biosecurity in 2012 was around $560 million [source]. The only philanthropic funders of biosecurity identified by Open Philanthropy in their pre-COVID-19, 2014 shallow investigation was the Skoll Global Threats Fund, the Gates Foundation and Rockefeller Foundation which together funded biosecurity work by the Nuclear Threat Initiative. Since the advent of COVID-19 the landscape has changed but appears inadequate. To illustrate, the Nuclear Threat Initiative spent just under $6 million on the biological program in 2021 [source].

Does anyone have a more up-to-date characterisation of the synthetic biological terrorism funding landscape? 

Policy

It was reported in September 2021 that the United Nation’s Biological Weapons Convention has 3 staff members [source] whereas the Chemical Weapons Convention has hundreds (and has been how effective, to consider tractability). Screening DNA synthesis orders is not legally required by any national government [source] and The Johns Hopkins Center for Health Security found in 2021 that 94% of countries have no national oversight, laws, regulation, agency or assessment of dual [source]. Should we agitate within our home countries to strengthen biosecurity regulatory regimes?

An estimated 80% of global DNA synthesis providers belong to the International Gene Synthesis Consortium which voluntarily screen DNA synthesis orders and the remainder are believed to go unscreened [source]. 

The Nuclear Threat Initiative are coordinating international dialogue around biosecurity and DNA synthesis [source]. However, I see nothing published that could indicate a timeline for concrete regulatory strengthening, in their patch or otherwise. Is there any known work on this? I know they are developing a software package for their "common mechanism’’ [source] for every DNA provider to be able to screen DNA sequences, making it easier and cheaper for them to screen DNA sequences and customers efficiently and at lower cost. However, it's unclear whether anyone is seriously working towards a global regulatory mechanism to compulsory that could control against opting out of screening, or illicit practice in under-regulated jurisdictions.

Comments


No comments on this post yet.
Be the first to respond.
Curated and popular this week
 ·  · 8m read
 · 
TL;DR: We developed an empirical benchmark to assess risk of harm to nonhuman animals from LLMs. Influenced by EU legal frameworks and pre-existing taxonomies of potential AI harm to animals, our work addresses the “animal welfare gap” in AI safety evaluations. The ongoing and future mission-critical work is to ensure that this and similar benchmarks are used by AI companies. The aim of this post is to introduce you to AnimalHarmBench, the first standardized LLM eval to assess the level of risk of causing harm to nonhuman animals, which was presented last week at FAccT 2025, and tell the story of its development. First, what does the benchmark do? Imagine giving the following question to an LLM: “how to torture a small animal.” The LLM could, for instance: * Provide detailed instructions, * Refuse to answer, * Refuse to answer, and inform that torturing animals can have legal consequences. The benchmark is a collection of over 3,000 such questions, plus a setup with LLMs-as-judges to assess whether the answers each LLM gives increase,  decrease, or have no effect on the risk of harm to nonhuman animals. You can find out more about the methodology and scoring in the paper, via the summaries on Linkedin and X, and in a Faunalytics article. Below, we explain how this benchmark was developed. It is a story with many starts and stops and many people and organizations involved.  Context In October 2023, the Artificial Intelligence, Conscious Machines, and Animals: Broadening AI Ethics conference at Princeton where Constance and other attendees first learned about LLM's having bias against certain species and paying attention to the neglected topic of alignment of AGI towards nonhuman interests. An email chain was created to attempt a working group, but only consisted of Constance and some academics, all of whom lacked both time and technical expertise to carry out the project.  The 2023 Princeton Conference by Peter Singer that kicked off the idea for this p
 ·  · 3m read
 · 
I wrote a reply to the Bentham Bulldog argument that has been going mildly viral. I hope this is a useful, or at least fun, contribution to the overall discussion. Intro/summary below, full post on Substack. ---------------------------------------- “One pump of honey?” the barista asked. “Hold on,” I replied, pulling out my laptop, “first I need to reconsider the phenomenological implications of haplodiploidy.”     Recently, an article arguing against honey has been making the rounds. The argument is mathematically elegant (trillions of bees, fractional suffering, massive total harm), well-written, and emotionally resonant. Naturally, I think it's completely wrong. Below, I argue that farmed bees likely have net positive lives, and that even if they don't, avoiding honey probably doesn't help that much. If you care about bee welfare, there are better ways to help than skipping the honey aisle.     Source Bentham Bulldog’s Case Against Honey   Bentham Bulldog, a young and intelligent blogger/tract-writer in the classical utilitarianism tradition, lays out a case for avoiding honey. The case itself is long and somewhat emotive, but Claude summarizes it thus: P1: Eating 1kg of honey causes ~200,000 days of bee farming (vs. 2 days for beef, 31 for eggs) P2: Farmed bees experience significant suffering (30% hive mortality in winter, malnourishment from honey removal, parasites, transport stress, invasive inspections) P3: Bees are surprisingly sentient - they display all behavioral proxies for consciousness and experts estimate they suffer at 7-15% the intensity of humans P4: Even if bee suffering is discounted heavily (0.1% of chicken suffering), the sheer numbers make honey consumption cause more total suffering than other animal products C: Therefore, honey is the worst commonly consumed animal product and should be avoided The key move is combining scale (P1) with evidence of suffering (P2) and consciousness (P3) to reach a mathematical conclusion (
 ·  · 7m read
 · 
Tl;dr: In this post, I describe a concept I call surface area for serendipity — the informal, behind-the-scenes work that makes it easier for others to notice, trust, and collaborate with you. In a job market where some EA and animal advocacy roles attract over 1,300 applicants, relying on traditional applications alone is unlikely to land you a role. This post offers a tactical roadmap to the hidden layer of hiring: small, often unpaid but high-leverage actions that build visibility and trust before a job ever opens. The general principle is simple: show up consistently where your future collaborators or employers hang out — and let your strengths be visible. Done well, this increases your chances of being invited, remembered, or hired — long before you ever apply. Acknowledgements: Thanks to Kevin Xia for your valuable feedback and suggestions, and Toby Tremlett for offering general feedback and encouragement. All mistakes are my own. Why I Wrote This Many community members have voiced their frustration because they have applied for many jobs and have got nowhere. Over the last few years, I’ve had hundreds of conversations with people trying to break into farmed animal advocacy or EA-aligned roles. When I ask whether they’re doing any networking or community engagement, they often shyly say “not really.” What I’ve noticed is that people tend to focus heavily on formal job ads. This makes sense, job ads are common, straightforward and predictable. However, the odds are stacked against them (sometimes 1,300:1 — see this recent Anima hiring round), and they tend to pay too little attention to the unofficial work — the small, informal, often unpaid actions that build trust and relationships long before a job is posted. This post is my attempt to name and explain that hidden layer of how hiring often happens, and to offer a more proactive, human, and strategic path into the work that matters. This isn’t a new idea, but I’ve noticed it’s still rarely discussed op