Thanks for asking.  I see a high degree of alignment and overlap between the EA community’s framing and cause areas, and NTI’s work.  At NTI, our vision is a world safe from preventable global catastrophe, and our mission is to transform global security by driving systemic solutions to nuclear and biological threats imperiling humanity.  It’s been exciting for me to get to know the EA community better and I look forward to exploring ways we can leverage each others’ strengths, capacities and skills to expand our collective impact.  At NTI, we have a team of some of the world’s best experts in nuclear and biosecurity risk reduction.  We are happy to provide our advice and expertise to members of the EA community who are interested in bio and nuclear risk reduction.  

Members of the EA community can help us build our capacity for action, through helping to build global awareness about the risks to humanity of catastrophic risks, by choosing careers in nuclear and bio risk reduction, by providing financial resources to institutions doing impactful work in this issue space, and through helping to build the political space for action.  After watching the nuclear and bio fields decline for several decades, it’s great to see the building momentum of effective altruism!  Many in our traditional NGO community were beginning to feel like a dying tribe trying to protect a small patch of the galaxy that humanity calls home.  I think the EA community represents the new vanguard, and a new generation of ideas and energy for problem solving associated with catastrophic threats to humanity.  We are excited to work together!

The reality is that as long as other states possess nuclear weapons, the United States will continue to maintain nuclear forces for deterrence.  I believe their purpose should be only to deter the use of nuclear weapons by others, and that the US should maintain the minimum number of forces we believe we need to serve that purpose.  At the same time, the US must vigorously pursue further nuclear reductions and limits with Russia, and eventually, multilateral arms control to reduce and ultimately eliminate the nuclear forces of all states that have them including China.  We must also strengthen the nuclear nonproliferation regime to prevent the emergence of new nuclear weapons states. It is essential to return to an agreement with Iran to make sure it does not develop nuclear weapons, and we need to reinvigorate diplomacy with North Korea to stop, reverse and ultimately eliminate its nuclear weapons program while providing security and economic incentives in return.    

For nuclear risk reduction, I believe AI/ML and the growing availability of big data are potentially game changing.  In a world where the vast majority of collective human actions and behaviors leave a digital footprint, we now have a set of new tools for detecting illicit nuclear behavior (like illicit trade in nuclear dual use components, or the production of weapons grade uranium or plutonium). Like any new technology, we have to work to capture the benefits while minimizing the risks they bring, so we at NTI are looking at both sides.

Last year NTI published a report on a pilot project we undertook with a partner, C4ADS, that demonstrated how we were able to use some large trade datasets and machine learning to detect illicit nuclear trade. After we published our work, a number of the entities we found were added to government sanctions lists. We believe this work can be significantly expanded to build and demonstrate an entirely new approach to a verification system– one that enables a safer nuclear strategy for the future.

In April, the National Academies published an Interim Report, (led by Jill Hruby) on their ongoing exploration of Nuclear Proliferation and Arms Control Monitoring, Detection, and Verification.  The Interim Report finds that technological advances provide unprecedented opportunities for staying ahead of complex and expanding monitoring, detection, and verification challenges; and that monitoring, detection, and verification technology should be a higher national security priority with a long-term vision and regular evaluation of progress.  Technological change is creating transformational possibilities for nuclear threat reduction and we should work diligently to develop national and global innovation ecosystems that can deliver on this promise.

On the flip side, AI/ML and automated decision-making also pose new challenges and risks for the nuclear system.  In August, NTI published a report titled Assessing and Managing the Benefits and Risks of Artificial Intelligence in Nuclear-Weapon Systems.  This report, by Jill Hruby, then Sam Nunn Distinguished Fellow (and now Administrator of the National Nuclear Security Administration) and former NTI intern and MIT doctoral student M. Nina Miller explores the possible applications of AI to nuclear-weapons systems and assesses the benefits, risks, and strategic stability implications.  The report recommends: 1) research on low technical-risk approaches and fail-safe protocols for AI use in high-consequence applications; 2) that states with nuclear weapons make declaratory statements about the role of human operators in nuclear-weapon systems and/or the prohibition or limits of AI use in their nuclear-weapon systems; and 3) increased international dialog on the implications of AI use in nuclear-weapon systems, including how AI could affect strategic and crisis stability, and explore areas where international cooperation or development of international norms, standards, limitations, or bans could be beneficial.

On the biosecurity side, we’re really excited about our work on a prototype to globally expand DNA synthesis screening practices and to prevent exploitation by malicious actors and misuse. 

NTI | bio, with the support of talented technical consultants and in partnership with the World Economic Forum, has developed several critical elements of the Common Mechanism prototype, which improves on current industry best practices. The Common Mechanism prototype includes: (1) novel databases of ‘benign’ and ‘biorisk’ sequences informed by publicly available literature and industry expertise and inputs and (2) and a screening algorithm to compare incoming orders with the contents of the aforementioned databases. A key feature of the algorithm is that it uses statistical models known as ‘profile hidden Markov models,’ which have been shown through early tests to be resilient against subversion attempts. 

As a next step, the project team will construct a “decision support system” which will present the results of the screening algorithm, while providing important context about the kind of organism the sequence comes from, the function it might have, and a recommendation regarding whether to proceed with the order. These tools will allow DNA synthesis providers to perform an effective and efficient assessment as to whether the DNA synthesis order in question is benign or constitutes a risk. 

Future steps include acquiring additional data sets to test the Common Mechanism prototype, improving the quality of the reference databases, and rigorously testing the Common Mechanism with close partners and a wider circle of other peer reviewers.

They are inextricably linked. The 2021 Global Health Security Index – a project led by NTI in partnership with the Johns Hopkins Center for Health Security –  found that 94% of countries have no national level oversight measures for dual use research, no agency responsible for the oversight, and no evidence of national assessments of dual use research. Additionally, there is no international entity that has dedicated, as its top priority, efforts to strengthen biosecurity and bioscience governance and to reduce emerging biological risks associated with technology advances. While the development of risk-reducing technologies is critical, to ensure these tools are applied and that actors are held accountable for reducing risk, we must also make progress on the policy front. 

NTI’s efforts to design and launch the International Biosecurity and Biosafety Initiative for Science (IBBIS) recognize the need to focus on both aspects of biosecurity, which is clear in the mission:  IBBIS works collaboratively to strengthen global biosecurity norms and develop innovative tools to uphold them. We undertake this work to safeguard science and reduce the risk of catastrophic events that could result from deliberate abuse or accidental misuse of bioscience and biotechnology.     

Thanks for this very thoughtful question, Tessa.

1. What are the most important problems for this project, and what is stopping you (/NTI) from working on them right now?

The Biosecurity Innovation and Risk Reduction Initiative (BIRRI) was born from the recognition that advances in biotechnology are rapidly outpacing the ability of governments to provide effective oversight. Through BIRRI, we are working to identify efforts that would be most impactful in safeguarding science and reducing the risk of catastrophic events that could result from deliberate abuse or accidental misuse of bioscience and biotechnology.  

The research ecosystem that enables biotechnological developments is diverse—involving publishers, academic scientists, private industry, research funders, bioethicists, managers, security experts, statisticians, and more. Although each group has a shared interest in the overall success of research, their near-term priorities may not always be aligned. BIRRI seeks to bring together these diverse stakeholders to identify and advance practical tools that, when paired with appropriate incentive structures, have a real chance at reducing risks. 

Projects under the umbrella of the initiative were collaboratively developed and are designed to cut across the research lifecycle, providing a layered defense against deliberate or accidental misuse of biotechnologies.

We believe all five of the initiatives you listed are critical components of this layered defense, and we’re working on each of these to different degrees. 

For example, NTI is partnering with Stanford University on the Visibility Initiative for Responsible Science in an effort to understand, improve, and standardize risk reduction practices among communities involved in research, with a particular focus on funders and publishers. 

We’re also partnering with the World Economic Forum and global leaders from industry, academic, philanthropy, and international organizations to develop an international Common Mechanism for DNA synthesis screening to protect this critical service from malicious actors who may seek to exploit it to cause harm.

To create systemic and sustainable change in the global biosecurity architecture, NTI and partners have also prioritized efforts to establish the International Biosecurity and Biosafety Initiative for Science (IBBIS), a new organization that will work collaboratively to strengthen global biosecurity norms and develop innovative tools to uphold them. The goal of IBBIS will be to safeguard science and reduce the risk of catastrophic events that could result from deliberate abuse or accidental misuse of bioscience and biotechnology.

IBBIS is slated to launch in 2022 and will begin with a narrow focus on improving DNA synthesis screening practices internationally with the flexibility to expand its remit over time to encompass, but also expand beyond, the other goals outlined in each initiative. 

We see our current approach as developing urgently needed, risk-reducing pilot efforts while simultaneously building out the larger institutional structure required to fully implement and sustain these efforts.  

2. What is the limiting factor on the project’s growth and progress?

We’re pleased with the progress of our efforts, however, ultimately, limited team resources – both in terms of human and financial resources - is a limiting factor through much of our work. There are many gaps in the international biosecurity architecture and many important solutions to develop. To make the best use of our limited staff time, we try to prioritize projects that are important, timely, and have the potential to advance biosecurity in the long term. 

Additionally, engaging stakeholders and building communities is crucial to our work. Strong relationships take time to build, but are essential to gaining trust, avoiding “one-off” solutions, and ensuring that our efforts create sustainable and durable positive change.

3. What problems in this project are the largest order of magnitude? What changes could you make that would result in a 100x or 1000x increase in this project’s positive impact?

Referring to a previous point about developing a common mechanism for screening DNA synthesis orders, creating a tool and set of standards to simplify screening is an important first step to increasing the depth and breadth of screening coverage.

Coupling the development of tools like the common mechanism with the launch and sustainment of a global organization like IBBIS, with the capacity to work with diverse partners to identify, socialize, and implement stronger biosecurity norms though both practical tools and accompanying incentive structures, will increase the positive impact of these efforts on risk reduction. This will require a careful design and approach and NTI is working with an international steering group to shape this effort. 

Thanks so much for this really fun question!  It provoked a really interesting discussion with some of my colleagues.

There are a large number of important works of fiction that illuminate nuclear dangers and different ways the human species can respond to them.  

A new personal favorite of mine is Kim Stanley Robinson's The Ministry for the Future, which is about climate change, but illuminates ways in which global governance might begin to value the lives of future generations who would suffer unbelievably from planetary scale harms.  Nuclear war poses an immediate danger of planetary scale harm, but human efforts to reduce the risk of this harm are hobbled by our institutions that privilege the security of nation states over the security of our species and planet.  The Ministry for the Future suggests discrete actions and pathways that we should explore to improve our response to planetary dangers for the sake of our descendants.

Another recent favorite is a new novel published earlier this year by Admiral Jim Stavridis and and Elliott Ackerman called 2034:  A Novel of the Next World War.  The novel isn’t really about resilience, but rather vividly illustrates just how easily festering political disputes and emerging technology could lead us to blunder into a nuclear war no one wants, but that kills many millions of people and from which humanity learns nothing that might reduce future risks.

I want to include a few other titles suggested by colleagues (and which I am adding to my own reading list as a result of our office discussion!):

A Canticle for Leibowitz explores the far future consequences of a nuclear war to illuminate both the ethical challenges we face today in relying on nuclear weapons for defense and the planetary scale of danger; M.K. Wren's A Gift Upon the Shore illuminates the inadequacy of our emergency planning and how the few survivors of a large nuclear exchange might succumb to the delayed political, economic, and social consequences over a few years, while Nevil Shute's On the Beach emphasizes how the environmental damage of a nuclear war could engulf the planet in a few months.

William Prochnau's 1983 novel, Trinity's Child, dramatizes the chaos that would immediately follow the first use of nuclear weapons, tearing to ribbons our detailed plans to prosecute an extended nuclear war and for continuity of government.  Prochnau vividly imagines what those who have played nuclear wargames know well:  nuclear escalation cannot be controlled.

H.G. Wells 1914 novel, The World Set Free, leveraged Frederick Soddy's textbook, The Interpretation of Radium, to foresee the possibility of atomic bombs (and to coin that name for these devices).  Wells foresaw that massive, instant, indiscriminate destruction would render what the world had known as war meaningless and impossible, but human beings could not see this “until the atomic bombs burst in their fumbling hands.”   Wells foresaw that “these bombs and the still greater power of destruction of which they were the precursors might quite easily shatter every relationship and institution of mankind.”   Before the end in which Wells magics the reader off to an age of unexplained global enlightenment that frees humanity to enjoy the benefits of limitless nuclear power, he burns countless cities instantly to ash and imagines an age of sickness, filth, starvation, and “everybody doing uncongenial things;” followed by “the pill-carrying age” to numb unbearable horror.

We need to do both – in parallel and with urgency – until we mitigate both risks. At NTI, we use a mix of strategies to maximize our effectiveness in each area. We engage with stakeholders and governments, we develop and field test innovative ideas, we convene and consult with the world’s best experts – and then we adjust our strategies as we go. Through it all, we use our evidence-based findings and solutions to keep the pressure on global leaders, whether in government or the private sector, to prioritize risk reduction in both areas. 

We have to pursue multiple strategies for reducing nuclear risks – no one strategy alone is sufficient. Because governments possess nuclear weapons and have the resources necessary for implementing risk reduction measures at the scale that’s needed, it’s imperative to continue to leverage policy change by governments as a core focus of nuclear risk reduction efforts. This includes generating creative ideas for policy solutions that governments could adopt. But a strategy that focuses only on persuading government leaders to adopt policy changes has proven to be insufficient on its own.  The adoption of nuclear risk reduction measures has recently been outpaced by growing risks, and policy makers’ attention to nuclear issues has been declining as the awareness and attention of global publics has also declined.

So we must also aim to drive a strategy of culture change by working to raise awareness of nuclear risks among broader segments of the public.  We have work to do to communicate more effectively about the risks of nuclear weapons to global publics.  The more awareness we can create, the more likely it is that governments will feel they have the political support (the Overton window) for reducing the threat of nuclear weapons.  

 Individuals can educate themselves on the dangers posed by nuclear weapons. There are many places to learn about nuclear risks and nuclear policy (including on NTI’s website!).  A few specific reading suggestions include:

•  2034: A Novel of the Next World War by Admiral James Stavridis;
• My Journey at the Nuclear Brink by former U.S. Secretary of Defense William Perry, and
• Command and Control by Eric Schlosser. 

These could be discussed in book groups or informal dinners with friends and neighbors. Individuals can then make their views known by communicating to political leaders their concerns about the danger of nuclear weapons and the need for governments to do more to reduce them. Even small actions, such as hosting a book group or dinner party and discussing these issues can go a long way to raise awareness.

In my view, the most immediate catastrophic/existential threats facing humanity are biological risks, nuclear risks,  climate change, and AI. 

To address these global catastrophic risks to humanity, the world must first and foremost recognize the enormity of these threats and the catastrophic consequences of not addressing them with the urgency or creativity required to solve them.  

Second, these threats have in common the reality that their mitigation will require unprecedented global cooperation and an approach that takes advantage of the expertise, perspectives and wisdom of the “whole of society.” No one state can mitigate these threats on its own or isolate itself from their consequences. 

Third, they all require radically new ways of thinking and working together to devise solutions. As humans we tend to think linearly, but we are entering an unprecedented period of disruption and discontinuity, where incremental, linear solutions are unlikely to produce the kind of change we need at the pace we need it.  We need to untether from the past in thinking about the future and remain open to novel solutions.

Fourth,  enhanced political will is required to do what is necessary to avert the catastrophic course the world is on in all four of these dimensions. The EA community can play a really significant role in building the political will for change that is so urgently needed.

Technology in the nuclear weapon space is quickly evolving. I can imagine numerous scenarios where emerging technologies could lead to nuclear escalation and devastating conflict. For example, there are plausible scenarios involving a conflict that evolves from a cyber attack or cyber interference in nuclear command and control systems. During a moment of high tension, such an attack could trigger a nuclear response. This could also happen due to a cyber attack from a nonstate actor. Confusion about intent and attribution could lead to a breakdown in nuclear deterrence. Nuclear-armed states cannot completely prevent such scenarios (short of nuclear disarmament), but they can reduce their likelihood by agreeing to rules of the road that would prohibit cyber attacks against nuclear weapons and command and control and warning systems. Moreover, nuclear-armed states could conduct their own “failsafe reviews” to consider additional steps they could take unilaterally or potentially cooperatively to reduce the risk of unauthorized or miscalculated use of nuclear weapons.

For more on the risks and need for a fail-safe review process, see the recent Washington Post op-ed by NTI Co-Chairs Ernie Moniz and Sam Nunn.