Researchers from Cambridge University's Centre for the Study of Existential Risk and Oxford University's Center for the Governance of AI at the Future of Humanity Institute submitted advice to the UN Secretary-General’s High-level Panel on Digital Cooperation.
The High-level Panel on Digital Cooperation was established by the UN Secretary-General in July 2018 to identify good examples and propose modalities for working cooperatively across sectors, disciplines and borders to address challenges in the digital age. It is cochaired by Melinda Gates and Jack Ma.
The full submission is below.
Authors: Dr Luke Kemp1, Peter Cihon2, Matthijs Michiel Maas2, Haydn Belfield1, Dr Seán Ó hÉigeartaigh1, Jade Leung2 and Zoe Cremer1. (1=CSER, 2=FHI)
International Digital Cooperation must be underpinned by the effective international governance of artificial intelligence (AI). AI systems pose numerous transboundary policy problems in both the short- and the longterm. The international governance of AI should be anchored to a regime under the UN which is inclusive (of multiple stakeholders), anticipatory (of fast-progressing AI technologies and impacts), responsive (to the rapidly evolving technology and its uses) and reflexive (critically reviews and updates its policy principles). We propose some options for the international governance of AI which could help coordinate existing international law on AI, forecast future developments, risks and opportunities, and fill critical gaps in international governance.
Digital cooperation will rise or fall by the use or misuse of rapidly developing artificial intelligence (AI) technologies. AI will transform international social, economic, and legal relations in ways that spill over far beyond the digital realm. Digital cooperation on AI is essential to help stakeholders build capacity for the ongoing digital transformation and to support a safe and inclusive digital future. Accordingly this submission will focus on the international governance of AI systems.
AI technologies are dual-use. They present opportunities for advancements in transport, medicine, the transition to renewable energy and lifting standards of living. Some systems may even be used to strengthen the monitoring and enforcement of international law and improve governance. Yet they also have the potential to create significant harms. These include labour displacement, unpredictable weapons systems, strengthened totalitarianism and destabilizing strategic shifts in the international order (Dafoe 2018; Payne 2018). The challenges of AI stem from both capabilities that already exist, or will be reached in the near-term (within 5 years), as well as from longer-term prospective capabilities. The two are intricately intertwined. How we address the near-term challenges of AI will shape longer-term policy and technology pathways (Cave and ÓhÉigeartaigh 2019). Yet the long-term disruptive impacts could dwarf other concerns. Both need to be governed in tandem.
Challenges from Existing and Near-Term Capabilities
Challenges from Long-Term Capabilities
While most of these challenges have not received sufficient attention, several have been mapped in The Malicious Use of Artificial Intelligence report (Brundage & Avin et al 2018), AI Governance: a Research Agenda (Dafoe, 2018), and in the Future of Life’s (2019) 14 policy challenges. Greater attention is needed to forecasting these potential challenges. Both the foresight of policy problems and the magnitude of existing issues underline the need for international AI governance.
There are already over a dozen sets of principles on AI composed by governments, researchers, standard-setting bodies and technology corporations (cf. Zeng et al. 2019). Most of these coalesce around key principles of ensuring that AI is used for the common good, does not cause harm or impinge on human rights, and respects values such as fairness, privacy, and autonomy (Whittlestone et al. 2019). We suggest that the High-level Panel on Digital Cooperation compile and categorise these principles in its synthesis report. Importantly, we need to examine trade-offs and tensions between the principles to refine rules for how they can work in practice. This can inform future negotiations on codifying AI principles.
The international governance of AI should also draw from legal precedents under the UN. In addition to general principles of international law, principles such as the polluter pays principle (those who create externalities should pay for the damages and management of externalities) could be retrofitted from the realm of environmental protection to AI policy. Values from bioethics, such as autonomy, beneficence (use for the common good), non-maleficence (ensuring AI systems do not cause harm or violate human rights), and justice are also applicable to AI (Beauchamp and Childress 2001; Taddeo & Floridi 2018). Governance should also be responsive of existing instruments of international law, and cognizant of recent regulatory steps by international regulators on the broader range of global security challenges created by AI (Kunz & Ó hÉigeartaigh 2019). Finally, while some specialization of AI governance regimes for distinct domains is unavoidable, steps should be taken to ensure these distinct standards or regimes reinforce rather than clash with each other.
International governance of AI should be centred around a dedicated, legitimate and well-resourced regime. This could take numerous forms, including a UN specialised agency (such as the World Health Organisation), a Related Organisation to the UN (such as the World Trade Organisation) or a subsidiary body to the UN General Assembly (such as the UN Environment Programme). Any regime on AI should fulfil the following four objectives:
The Panel should consider the following options as components for an international regime:
The outlined options for a regime should be anticipatory, reflexive, responsive and inclusive. This adheres to the key tenets of Responsible Research and Innovation suggested by scholars (Stilgoe et al 2013). To be inclusive we suggest following the ILO’s innovative model of multipartite representation and voting. In this case voting rights could be distributed to nation states as well as other critical stakeholder group representatives. An ability to anticipate emerging challenges and respond to the quickly evolving technological landscape would be enabled by the IPAI. Responsiveness could be built into the body by having principles on AI reviewed and updated every three years. This would ensure that policies reflect the latest and in-country experiences.
With prudent action and foresight, the UN can help ensure that AI technologies are developed cooperatively for the global good.
Beauchamp, T. and Childress, J. (2001). Principles of biomedical ethics. Oxford University Press, USA.
Brundage, M. and Avin, S. et al. (2018). The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation. Future of Humanity Institute and the Centre for the Study of Existential Risk.
Cave, S. and ÓhÉigeartaigh, S. (2019). An AI Race for Strategic Advantage: Rhetoric and Risks. In AAAI/ACM
Conference on Artificial Intelligence, Ethics and Society.
Cave, S. and ÓhÉigeartaigh, S. (2019). Bridging near- and long-term concerns about AI. Nature Machine Intelligence, 1: 5-6
Dafoe, A. (2018). AI Governance: A Research Agenda. Future of Humanity Institute, Oxford University.
Guterres, António. “UN Secretary-General’s Strategy on New Technologies.” United Nations, September 2018. http://www.un.org/en/newtechnologies/images/pdf/SGs-Strategy-on-New-Technologies.pdf.
Kunz, Martina, and Seán Ó hÉigeartaigh. “Artificial Intelligence and Robotization.” In Oxford Handbook on the International Law of Global Security (Forthcoming), edited by Robin Geiss and Nils Melzer. Oxford University Press.
Payne, K. (2018). Artificial Intelligence: A Revolution in Strategic Affairs? IISS.
Stilgoe, J., Owen, R. and Macnaghten, P. (2013). Developing a Framework for Responsible Innovation. Research Policy, 42(9): 1568-1580
Taddeo, Mariarosaria, and Luciano Floridi. “How AI Can Be a Force for Good.” Science 361, no. 6404 (August 24, 2018): 751–52. https://doi.org/10.1126/science.aat5991.
Whittlestone, J., Nyrup, R., Alexandrova, A. and Cave, S. (2019). The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions. Proceedings of the 2nd AAAI/ACM Conference on AI, Ethics, and Society. AAAI and ACM Digital Libraries.
Zeng, Yi, Enmeng Lu, and Cunqing Huangfu. “Linking Artificial Intelligence Principles.” Proceedings of the AAAI Workshop on Artificial Intelligence Safety (AAAI-Safe AI 2019), 2019. http://arxiv.org/abs/1812.04814.
This post and CSER's other advice post made me wonder how well one can gauge the effect of providing guidance to large governmental bodies.
For these or any past submissions, have you been able to gather evidence for how much CSER's advice mattered to an entire panel (or even just one member of a panel who took it especially seriously)?
Another question: Are any organizations providing advice to these panels that directly contradicts CSER's advice, or that seems to push in a bad or unimportant direction? It's hard to tell how much of this is "commonsense things everyone agrees on that just need more attention" vs. "controversial measures to address problems some people either don't believe in or think should be handled differently".
(Edit: Disclosure: I am executive director of CSER)
Re: your second question, I don't personally have a good answer re: bad advice - as these get hundreds of submissions I haven't read all or even most. (I do recall seeing some that have dismissed or ridiculed AI xrisk as conceptually nonsensical.)
Submissions I've been involved in tend towards (a) summarising already published work (b) making sensible, noncontroversial recommendations (c) occasionally gently keeping the overton window open (e.g. 'many AI experts think AGI is plausible at some point in the future, but on a very uncertain timeline; we should take safe development seriously and there is good work that can be done, and that is being done, at present on technical AI safety', as opposed to 'AI xrisk is real, scary and imminent)'. The aim of (c) being typically to counterweigh the 'AI safety/alignment is nonsense and everyone working on it is deluded' view rather than to promote action.
There are a few reasons for this. These open calls for evidence are noisy processes, and not the best way to influence policy on controversial topics or in very concrete ways. However, producing reputable input for them is a good way to get established as a reputable, trustworthy expertise source and partner. In particular, my impression is that it allows people in government, including those already concerned with these issues, greater scope to engage with orgs like ours in more in-depth conversation and analysis (more appropriate for the 'controversial/concrete action-relevant' engagement). It's easier to justify investing time and resources in an org that's been favourably featured in these processes as opposed to 'random centre somewhere working on slightly unusual topics'. But it can be hard to disentangle exactly how much these submissions play a role, versus Cambridge/Oxford 'brand', track record of academic success and publications, 1-1 meetings with policymakers that would have happened anyway, etc.
(Edit: Disclosure: I am executive director of CSER)
Thanks for good questions. These 2 submissions are very recent, so little time to demonstrate follow-on influence/impact. Some evidence on this and previous submissions that indicate work was likely well-received/influential:
Thanks for both of these answers! I'm pleasantly surprised by the strength and clarity of the positive feedback (even if some of it may result from the Cambridge name, as you speculated). I'm also surprised at the sheer number of submissions to these groups, and glad to see that CSER's material stands out.
Thanks Aaron!
Most of our submissions are in collaboration with other leading scholars/organisations, e.g. FHI/GovAI and CFI, so credit should rightly be shared. (We tend to coordinate with other leading orgs/scholars when considering a submission, which often naturally leads to joint submission).
These are good questions, thanks Aaron. A quick placeholder to say that I'll give an answer (from my personal perspective) tomorrow. (Haydn may also have comments on, and evidence relating to, this).