How AI could create the world’s biggest problems

How advanced AI could pose the world's most pressing problems

For over a decade, we've looked into the biggest problems in the world and approaches to solving them. We think the cluster of challenges raised by advanced AI are the most pressing problems facing humanity today — given their scale, urgency, and the promising but neglected opportunities to address them.

Our concern about AI risks isn't a reaction to the surge of interest in AI since the 2022 release of ChatGPT. We've argued that AI could pose catastrophic risks since 2016, and others raised related concerns long before then (1, 2, 3, 4).

In short, we think it's plausible advanced AI could radically transform the world. This could pose extreme challenges for humanity, and it presents a potentially unique opportunity for having a positive impact.

We go through the specific challenges we think are most pressing in our problem profiles. This article explains why advanced AI gives rise to such important issues in general.

There are a lot of arguments you could make here — like the argument that advanced AI will constitute a "second species" or that AI will make the 21st century "the most important century" for humanity.¹

But here's the argument that makes the risks feel especially compelling to us:

1. AI could replace human labour in some of the most economically valuable fields.
2. Replacing human labour in these fields could trigger the next radical transformation of society.
3. This transformation could be extremely rapid and dramatic, especially if there are fast feedback loops in AI R&D.
4. A rapid, AI-driven transformation would raise a range of major challenges, including existential risks.
5. Work on these challenges is tractable but neglected.

We'll argue for each of these claims below.

To be clear, you don't have to believe in this particular argument to think that AI poses existential threats. Even if AI doesn't automate lots of human labour, malicious actors could still use AI to design novel bioweapons or carry out sophisticated cyberattacks. And even without society being broadly transformed, advanced AI systems could still engage in deceptive behaviour and undermine humans. These issues in themselves could be enough reason to prioritise working on specific AI risks.

But the story we tell below, with the world rapidly being transformed through widespread automation, is the backdrop against which we expect all these risks to play out — a backdrop that makes them more likely to happen and potentially much harder to deal with.

After all, AI systems have a better chance of mobilising the resources needed to disempower humans if we've already deployed them in crucial roles in the economy. And the risks of malicious actors misusing AI are much greater if, through automating scientific research and technology development, powerful new weapons can be built faster than we can create robust protections against them being misused.

With this rough story in mind, it's easier to understand why so many unprecedented challenges could emerge around the same time, and have unusually serious consequences.

1. AI could replace human labour in the most economically valuable fields

Many technologies — like cryptocurrency, NFTs, the 'internet of things,' fusion, and quantum computing — have been overhyped. People often have high expectations of how much a new innovation will change the world, and reality sometimes falls short.

But we think AI is going to be different.

That's because, unlike other technologies, AI has the potential to compete with — and even go beyond — general human intelligence. And that means it could replace and reproduce the main driver of progress in our history: flexible human labour.

Some technologies, like ATMs, mimic extremely limited forms of human labour. Others, like steam engines and computers, also amplify what humans can do. But the idea behind artificial intelligence is that it'll be able to do almost any work humans can do — and do so mostly autonomously.

ATMs didn't make all the bank tellers unemployed² because there were other tasks the humans could easily shift into. But imagine an ATM that could not only hand out cash, but also manage the bank's IT systems, contribute to company strategy, and give customers tailored financial advice. Imagine it could do this mostly without our help, and more cheaply than human workers would. If that were the case, it's not clear why the bank would keep humans employed at all.³

Now suppose the same system that did all this for the bank could also do equivalent work for tech companies, scientific research labs, consultancy firms, think tanks, The New York Times, the US government, and so on.

That's the prospect raised by AI.

We're already seeing glimpses of AI's increasingly general ability to do human work. AI systems today can do things that just a decade ago would've been astonishing.

For example, consider the rapid progress language models have made on the GPQA benchmark, which asks challenging, PhD-level questions about chemistry, physics, and biology. In mid-2023, frontier AI performance was just slightly better than random guesswork on these questions. But since early 2025, many models have been outperforming human experts — and sometimes by a large margin.

They've also shown impressive improvement on software engineering tasks. For example, Anthropic's agentic coding tool Claude Code enables users to build applications by describing what they want — even if they have no coding experience.

A senior engineer at Google reported that Claude Code took one hour to generate a prototype of a system her team had spent a year exploring approaches to building. And Anthropic built its 'Cowork' product (a more user-friendly version of Claude Code for non-developers) in under two weeks by getting Claude Code to write most of the code.⁴

Current AI systems can also:

• Predict complex biomolecular structures and interactions: Google DeepMind's AlphaFold 3 — a successor to a Nobel Prize-winning AI system — can predict how proteins interact with DNA, RNA, and other structures at the molecular level.
• Solve hard maths problems competitively: Multiple AI models have reportedly achieved gold medal performance in the International Mathematical Olympiad. Separately, when 30 top mathematicians were challenged to devise problems they believed AI wouldn't be able to solve, OpenAI's o4 mini thwarted many of their best attempts — even solving one PhD-level question in about ten minutes.
• Improve robotics: Many leading robotics models are now AI-driven. For example, Boston Dynamics is enhancing its Atlas robots with Google DeepMind AI to help them better understand and manipulate their environments. These robots will be deployed for industrial work at Hyundai factories.
• Carry out extended tasks independently on your computer: Unlike earlier models that could only generate text, new 'agentic' AIs like Claude Code and OpenAI's Codex can now use many tools on your computer, execute code, search the web, and chain together multiple steps — allowing them to complete extended, real-world tasks with far less human involvement.
• Help with AI development itself: There's also evidence that AI systems can outperform humans in AI R&D tasks, at least when limited to a two-hour time window.
• Do much more.⁵

There are still plenty of things AI systems can't reliably do — especially most work that takes days or longer to complete — but the list of things these systems can't do is diminishing, and the pace of AI progress has been impressive.

Even with the range of capabilities they have today, it seems clear that AI systems could have considerable effects on society. At the very least, automating the specific tasks that AI is already good at — for example, in software engineering, biochemistry, and robotics — will speed up some areas of scientific progress and contribute to economic growth.

But we expect that AIs will become much more widely capable than they are today, and have far more transformative effects. A common adage in the industry is "today's AI is the worst AI you will ever use."

In fact, many people in the field think that AI will get good enough to do essentially anything that humans can do — and more.

One milestone here would be developing artificial general intelligence (AGI). People use this term in many different ways, but we'll use it to describe AI systems that can compete with humans on almost all cognitive tasks, or at least the most economically valuable ones — think advanced scientific research, designing new technologies and products, running businesses, consulting, and so on.⁶ This is the kind of system leading AI companies are actively trying to build, and they're funnelling billions of dollars into being the first to get there.

Looking at recent trends in AI development, we think it's surprisingly plausible (though far from guaranteed) that we'll get this sort of AGI within the next decade.

But it probably won't stop there. There's no reason to think that humans represent the ceiling of mental ability — so eventually, AI could greatly exceed human performance on many (if not all) cognitive tasks. Plausibly, they could even do work that's as far beyond human abilities as calculus is beyond chimpanzee abilities.⁷

It also might not take long before society makes giant advances in robotics. Although today's robots are very rudimentary, they're improving. And as our AIs get cognitively smarter, they'll also get better at both controlling robotic limbs and designing them. This means AI systems might quickly become able to outperform humans on many physical tasks as well.

Over the next few sections, we'll explain how the advanced AIs of the future could transform society and present serious risks.

Our argument focuses on the prospect that humanity develops AGI or something similar. This isn't the only important milestone (see below). But we think that if AI can match human abilities at the cognitive tasks that most drive innovation and economic production, that's likely enough to enable the rapid progress we describe in the following sections.⁸ And if AI becomes even more impressive than this — which we think is probable — the effects could be even more dramatic.

2. Replacing this much human labour could trigger the next radical transformation of society

So what would it mean if AI systems could outperform humans on such a wide range of tasks?

The first thing people often think of here is widespread unemployment. This is a serious possibility, and would have severe consequences for society.⁹ But we think focusing on it is actually missing an even bigger story.

A world with machines that can replace this much human labour would look so dramatically different that it can be hard to imagine.

For some sense of comparison, think of how different the world is today to how it was for our ancestors 200 years ago, 2,000 years ago, or 20,000 years ago. The worlds before electricity, or the printing press, or agriculture literally looked quite different, and they had entirely different ways of life.

With each of these major breakthroughs in technology, the world has been transformed.

Take the first Agricultural Revolution. Before agriculture, humans were mostly hunter-gatherers and often lived in small bands.¹⁰ The development of farming technologies like ploughs allowed us to produce far more food per person, leading to the first cities. And, to an increasing extent, some people could specialise in tasks other than finding food — which allowed humans to invent metalwork, writing, and early governance systems.

The Industrial Revolution followed a similar pattern. The arrival of technologies like the steam engine dramatically increased productivity — and sparked innovations in manufacturing and communication. Once again, this led to radical changes in how humans live: goods that were once luxury items became available to ordinary people, railways connected distant cities, and huge swathes of people shifted from rural to urban life.

What's going on here?

Each period of transformation in history has its own complex story, and there are competing theories about what drove them. But one popular explanation says we keep seeing the same rough pattern: powerful new technology both enables us to sustain larger populations and lets people do more with the same bodies and minds. This means more human labour and greater productivity — which has compounding effects, as it leads to a wave of even further innovation.¹¹

Since innovation often feeds economic growth, humanity has also become much wealthier in this process. In fact, since the late stages of the Industrial Revolution, we've seen roughly exponential growth in GDP¹² — though this is only an imperfect indicator of how much change has happened qualitatively.

A common thread in all these stories is that it seems growth has always been reliant on human labour — society has only been able to progress as fast as humans can produce and implement new ideas (that is, new theories, inventions, ways of working, and so on).

But we're now on the brink of a new breakthrough.

If future AIs can replace human workers in the most economically valuable fields, we'll no longer be so reliant on human labour to sustain these cycles of compounding innovation and wealth. Instead, AI could become the primary driver of progress.

We think this could lead to another transformation of society.

Like other technological breakthroughs, it could enable society to produce far more ideas and (perhaps) far greater economic output, fundamentally changing what the world looks like. But unlike previous technologies, AIs could actually take over the processes that most drive innovation and economic production (including the process of designing better AIs). And as we'll discuss next, these 'AI workers' could also have huge advantages over their human counterparts.

This could mean the transformation brought about by AI is extremely rapid, and more dramatic than anything we've seen before.

3. This transformation could be extremely rapid and dramatic

So what could happen as AIs automate more and more of the economy?

At the very least, we expect to see the total amount of labour quickly increase — since, unlike humans, AI systems can be easily copied at scale, given enough hardware.

Let's say we build an AI that could replace a human engineer. Estimates suggest huge uncertainty here, but running anywhere between thousands and hundreds of millions of copies of this AI at once may be feasible, depending on the circumstances.¹³

And this number could grow fast. With efficiency improvements to the algorithms behind these AI workers, we'll be able to run a greater number of copies with the same amount of compute.¹⁴ We might also be able to allocate more compute to running copies, by buying more chips or designing more efficient ones. Soon, we could have an AI workforce the size of a significant fraction of the world's working-age population.¹⁵

AI workers could also have other advantages over human workers:

• AIs can work much faster than humans, often compressing many hours of information processing into minutes.
• AIs may be able to coordinate far more efficiently between themselves than humans do — perhaps at lower costs and greater scales.
• AIs can become specialised very quickly, with different versions fine-tuned to be exceptionally good at specific tasks.

Based on these advantages alone, we could soon be seeing unprecedented levels of innovation and production as more work is performed by AIs. This could transform society — for the same reasons automating physical labour did during the Industrial Revolution.

And we think things could be even faster and more dramatic than you might expect based on the above.

That's because, at some point in this story, we expect AIs will be used to automate AI research and development itself. And this might trigger an "intelligence explosion": a period of rapid technological progress driven by AI systems that can create better AI systems.

Here's how it could unfold:

• AI systems become good enough to automate all or most work in AI R&D.
• These AI workers help us build better AI systems much faster.
• These better systems are then even more useful for automating AI R&D, which lets us build even better systems, and so on.

If this happens, it could create a positive feedback loop in which AI systems get better and better — possibly over a very short period of time.¹⁶

And this wouldn't just mean building AI systems that are better and better at AI R&D. It would mean speeding up improvements to AI capabilities more broadly, giving us increasingly capable and general AI workers to deploy across the wider economy — which, in turn, could accelerate progress in most areas of society.

What would "accelerated progress" look like?

As we've said, previous periods of transformation in history were ultimately limited by the pace at which humans could produce and implement new ideas. But now imagine having a vast workforce of AIs that can produce far more brilliant ideas than us, much faster than we could before — and act on them more efficiently.

We think this could transform society over a shorter timeframe than we've ever seen.

What would this even look like? For one thing, scientific discoveries could be made at unprecedented speed. The market could suddenly be flooded with new technologies that would otherwise have taken decades to develop. Infrastructure and manufacturing could expand to scales we can barely imagine today. More speculatively, if AI workers are deployed more widely, we could see a surge of fresh ideas — not just in science and technology, but in art, politics, philosophy, entertainment — that fundamentally change how we even think about the world.

The world could get much richer, too, since many innovations can increase economic production. In fact, some researchers think an influx of new ideas from AI workers would lead to "explosive" economic growth — and in turn, some of this new wealth could be used to accelerate idea production even further.

We're not sure we'll actually see an explosion in economic growth, since there could be bottlenecks on turning innovation into increased GDP. But we think there's at least a decent chance this will happen. And more importantly, even without explosive economic growth, there could still be a radical, qualitative transformation of society and our ways of life — which is what matters most to our story.¹⁷

How quickly could society be transformed, exactly? There will ultimately be some constraints on the trajectory of human progress. For example:

• At some point, we'll hit bottlenecks on AI development — e.g. the availability of compute, energy, or high quality data — that limit how much better AI workers can get over a short period of time.
• In every field, making progress could get increasingly difficult as AI workers quickly exhaust the low-hanging fruit of discoveries and new ideas.¹⁸
• If some types of work are particularly resistant to being automated — like complex physical tasks, or certain legal or political processes — this could also slow down the flywheel.

But even after accounting for these effects, some researchers still argue that the effects of AI automation could compress a century worth of progress into a decade.

This level of progress couldn't be sustained forever. But the world could already have been radically reshaped by the point things slow down — like how the Industrial Revolution eventually came to an end, but left behind a world that was totally unrecognisable.

4. A rapid, AI-driven transformation would raise a range of major challenges, including existential risks

The idea that AGI could supercharge innovation and economic output could be worth celebrating. The world could become extraordinarily rich, and we could rapidly develop new technologies that help us tackle the climate crisis or eradicate diseases.

Indeed, the promise of the technology is one reason why we expect some people to be excited about developing advanced AI systems. As Dario Amodei (CEO of Anthropic) puts it: a big motivator of AGI development is "a genuinely inspiring vision of the future".

Generally speaking, fears of emerging technology are often unjustified. Many innovations that have been viewed with suspicion, like vaccines and railways, have ended up being hugely beneficial for humanity.

But in this case, things seem different. For the first time, we're designing a whole new population of highly intelligent beings — agents that can do the most economically valuable things human minds can do, and might not rely on humans to do them.

This introduces complex dynamics we don't seem prepared to deal with and don't even fully understand. Humans navigating advanced AI could be like toddlers trying to navigate a world of adults, with changes to everything we know — in science, the economy, geopolitics, and even our ways of life — happening faster than we can get to grips with them.

Given the uncertainty around how AI development will unfold, it's hard to predict exactly what challenges we'll face. But the ones that seem most worrying to us are:

• We'll encounter agents that could be much smarter than humans, and might have goals of their own. Those goals might lead them to undermine human interests or even disempower humanity if we can't control them.
• Small groups could gain unprecedented power. If elite groups can control powerful AI, they'll be far less reliant on humans to get things done. With a vast AI workforce, they could amass previously unseen levels of economic and political influence, or even seize power — and probably wouldn't have strong incentives to represent the interests of the broader population.
• Dangerous technologies, like bioweapons, could become much more accessible. Access to highly capable AIs could make it much easier to design or get hold of dangerous weapons, significantly lowering the bar for people to cause devastating harm to humanity.
• We may create a large new population of beings whose welfare and interests matter, raising complicated questions about how to coexist with them.
• These factors may drive conflict and unrest, possibly culminating in a great power war or creating other unforeseen challenges.

How we navigate these dynamics could determine whether the future goes well or badly.

If we handle things wisely, we could create a flourishing future with unprecedented prosperity for all sentient beings, and could even spread to the stars. But if we lose control of advanced AI, or if bad actors use it to undermine the rest of the world's interests, we could face a catastrophe — like humans permanently losing our ability to shape the future, or going extinct.

In other words, we think these issues have existential stakes, making them among the most pressing problems in the world.

And although we're hopeful that these issues are tractable, we can't just assume our institutions will navigate them well by default. After all, this is confusing, unprecedented territory. And we've seen society stumble into disaster when facing new challenges we haven't sufficiently planned for — just think about the slow institutional responses to early COVID-19 warnings, or the numerous close calls we've seen with nuclear weapons.

The speed of this transition could matter a lot

There are two ways speed can matter critically to this transition:

• It matters how much time we have from now until we have extremely capable and general AI systems
• It matters how quickly the world is radically transformed by these systems once they arrive

If there's only a few years (or less!) until we get AGI or something similar, then we have limited time to avert the risks.

And although AI will still pose serious challenges even if it changes the world more slowly, a very rapid transformation would give us much less time to adapt to the changing circumstances and make wise decisions.

Even now, our institutions sometimes act too slowly — for example, it took around 50 years from the initial scientific warnings about global warming for the milestone Paris Climate Agreement to be signed. Unless we make big changes to how our institutions work, if AI becomes rapidly more capable and more productive, it seems it will be extremely difficult for society to keep up.

There is lively debate over how soon advanced AI systems might arrive and how quickly they might change the world. But there's at least a decent chance that they will be here within the next decade and things will change very fast — indeed, the level of expert concern suggests we need to take this possibility seriously (1, 2, 3). And given the stakes here, we think it's important to prepare for this possibility even if there's only a small likelihood — like a 10% chance — of it coming true.

This means we can't just ignore the risks or delay acting on them. We need to find robust solutions before it's too late.

5. Work on these problems is tractable but neglected

We've been helping people who want to work on this problem for over a decade. In this time, the field has grown substantially.

A 2025 analysis put the total number of people working on existential risks from AI at 1,100 — and we think even this might be an undercount, since it only includes organisations that explicitly brand themselves as working on 'AI safety.'

We'd estimate that there are actually a few thousand people focusing their work on the most important risks raised by AGI. But to put that into perspective, Nature Conservancy alone has 3,000–4,000 employees, and it's just one of many organisations working on environmental protection and climate change. Other global issues like public health also receive a lot of attention — for example, the World Health Organisation employs over 8,000 people.

This means AI risks are severely neglected in comparison to many other world problems — so each additional person working to address them can make a bigger difference.¹⁹

We're also optimistic that we can make progress on these problems. After all, humans are choosing to design and deploy these technologies, which means we have some influence over how things go.

Part of the challenge here is that the people who currently have the most influence over AI development aren't necessarily incentivised to prioritise safety. AI companies want to make money and face pressures to develop technologies quickly, without fully accounting for the risks they impose on society. Political leaders care about public opinion and election cycles, which gives them less time and motivation to focus on serving broader or longer-term interests. So we need people who want to prioritise using their careers to help others to work on the major challenges that might otherwise be ignored.

There are lots of ways you can help tackle these challenges. Check out our hub of AI career resources for more.

Objections and replies

What's next?

Inspired to work on addressing the risks from advanced AI?

Our job board features opportunities in AI technical safety and governance.

View all opportunities

Learn more

• Preparing for the intelligence explosion by Will MacAskill and Fin Moorhouse
• How AI-driven feedback loops could make things very crazy, very fast by Benjamin Todd
• The Most Important Century, a series by Holden Karnofsky and other authors
• Could advanced AI drive explosive economic growth? by Tom Davidson
• AI 2027 by Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, and Romeo Dean (we're sceptical that things will play out quite as quickly as in the scenario this describes — and in fact, the authors have published an updated model with more modest predictions of when AI will reach certain capabilities milestones)
• Our AI guide summary
• Will we have AGI by 2030? by Benjamin Todd
• When do experts expect AGI? by Benjamin Todd

Read more about AGI careers

Acknowledgements

Many thanks to Cody Fenwick, who drafted an earlier version of this article (much of which was incorporated here).

Thanks also to Arden Koehler, Adam Bales, Andreas Mogensen, Benjamin Todd, Niel Bowerman, and Aaron Gertler for input.