Hide table of contents

Last year, the Future of Life Institute organized the "AI Worldbuilding Contest", where they challenged competitors to envision a future where humanity has safely navigated the development of artificial general intelligence, avoiding major wars and catastrophes.  The winning entries were imaginative and diverse; I profiled a few of my favorite entries here.

Now, FLI is following up on the project with "Imagine A World"!  This new podcast series digs deeper into the eight winning entries, exploring the different ideas and solutions that each team used to build their distinct, aspirational futures.

Here's a little bit about each of the first four episodes; I'll cover the next four once they all come out!

 

"What if new governance mechanisms helped us coordinate?"

(Apple Podcasts, Soundcloud, Spotify, and here is the original worldbuilding entry)

This is the first episode of FLI's series, and it's also my own team's entry!  Here are some of my favorite moments from the podcast interview:

  • At the 11:15 mark, we introduce the overall historical arc of our scenario, then move on to talk about how new institutions like prediction markets and charter cities could improve humanity's ability to make wise decisions and coordinate on global problems.
  • At 27:12, talking about how a rapidly-changing landscape of AI capabilities will create a lot of uncertainty about who the ultimate "winners" vs "losers" will be.  This uncertainty might actually make international coordination easier, creating a situation like the "veil of ignorance" thought experiment from philosopher John Rawls.
  • Around 38:30, Holly and I gush over some of sci-fi stories that we took inspiration from -- books like Ada Palmer's Too Like The Lightning, Ursula LeGuin's The Dispossessed, and Yudkowsky's Inadequate Equilibria,  that take improvements in "social technology" just as seriously as advancements in science and engineering.

 

"What if narrow AI fractured our shared reality?"

(Apple Podcasts, Soundcloud, Spotify, and here is the original worldbuilding entry)
This world, from Michael Vassar, brings a strong dose of Robin-Hanson-esque pessimism about human motivations and realpolitik about the structure of societies and institutions.  In this world, cutting-edge AGI hasn't yet been rolled out to the world, instead being cautiously tested and tweaked for alignment.  But meanwhile, despite advancing living standards, society continues to go off the rails in the sanity department -- nations' state capacity is eroding, society is more politically fractured, and individual people have a more confused view about what's really important.

  • At 7:25, Vassar talks about his conception of how the world might break apart into isolated "media bubbles" where AI-influenced media leads people to become less and less connected to a shared sense of reality.  Plus a wacky analogy between AI and the movie "Who Framed Rodger Rabbit"!
  • At 17:36, we explore some of the social & economic details of Vassar's scenario, seeing how the world has gone off the rails in various ways.  Ultimately, this pessimistic take on human institutions seems aimed at investigating the same question that I tried to grapple with in my own scenario -- how can our current level of civilization, which feels so inadequate in terms of its capability for wise, coordinated action, ever hope to deal with the challenge of aligning and safely deploying a superintelligent AI?
  • At 39:07, some interesting thoughts about optimism vs pessimism, and why people are often biased towards being irrationally pessimistic.  This continues at 45:45, with discussion about a special kind of nihilism that Vassar sees emerging in today's social media culture.

 

"What if we had digital nations untethered to geography?"

(Apple Podcasts, Soundcloud, Spotify, and here is the original worldbuilding entry)

This team is based in Kenya, and their scenario is a deep dive into the possibilities for virtual communities and network states!  In their world, deep-learning-based AI techniques like large language models start to stall out in the near future; instead, AGI is ultimately achieved via "digital person" algorithms designed more closely to mimic the architecture of the human brain.

Here are my top highlights!

  • At 7:45, Tracy Kamande talks about how trying to imagine realistic futures -- and in the process, taking emerging technologies and other changes more seriously -- can help you live more intentionally, learning more and getting a more grounded view of what's going on in the world.
  • At 35:30, the concept of "digital nations", or "network states" as described in the book by Balajis Srinavasan.  Interestingly, over the past year, the pacific-island nation of Tuvalu has taken steps towards becoming a true network state, which it hopes will help it maintain sovereignty and provide services to its citizens even as rising sea levels erode the physical integrity of its territory.
  • At 43:00, a description of how they envision the details of "digital people", which need to be raised almost like human children (reminding me of the Ted Chiang story "The Lifecycle of Software Objects").  I especially appreciate the thought they put into constraining the nature of "digital people" (like the idea that digital people might be intrinsically averse to recursive self-improvement, or might only be able to process information at some fixed maximum speed) in order to avoid scenarios that would lead to "runaway computation explosion".  These constraints are needed for the story to work, since (like Holden Karnofsky outlines in this 80K interview) human-level AGI based on today's deep-learning architectures might still lead to explosive growth even if we never got to superintelligence.

 

"What if we designed and built AI in an inclusive way?"

(Apple Podcasts, Soundcloud, Spotify, and here is the original worldbuilding entry)

This episode features Elaine Czech and Vanessa Hanschke, both academics at the University of Bristol.  This team put a lot of thought into how new technologies might integrate into people's daily lives, and has a left-wing political perspective that emphasizes the importance of listening to marginalized voices in order to build a bright future for everyone.  Highlights from this interview:

  • Around 9:18 -- In this scenario the most advanced AGI models are large, costly, and unwieldy, so interaction mostly takes place in defined settings with an "oracle" consulting-like atmosphere.  True AGIs are heavily regulated so they are not commonly interacted with by the public.
  • At 20:39 -- This world features technological unemployment where most manual labor jobs disappear.  Despite this, human nature is slow to change. One imaginative way that society addresses this is by creating a virtual high-stakes war/competition space so  countries have an outlet to continue fighting.

 

Future Episodes

I've only caught up to about halfway through the series, but they've currently released seven out of eight total episodes!  You can subscribe to the Future of Life Institute podcast here -- besides these "Imagine A World" interviews, they also host talks with all sorts of fascinating intellectual figures on new ideas in philosophy, AI alignment, and other important topics.

Finally, the Future of Life Institute would like to note that they're not endorsing any one idea with this AI Worldbuilding Contest. Rather, they hope to grow the conversation about what futures people get excited about.

(Personally, I'm not an FLI employee or anything, and I  would like to note that I'm totally endorsing one idea, namely my own!  As I tried to showcase in my contest entry, I think that experimenting with improved institution designs like futarchy, network states, liquid democracy, Georgism, quadratic funding, etc, is an underrated approach within EA and has a lot of potential to create a safer, wiser, more flourishing civilization.  But that's a topic for another post!)

Comments


No comments on this post yet.
Be the first to respond.
Curated and popular this week
abrahamrowe
 ·  · 9m read
 · 
This is a Draft Amnesty Week draft. It may not be polished, up to my usual standards, fully thought through, or fully fact-checked.  Commenting and feedback guidelines:  I'm posting this to get it out there. I'd love to see comments that take the ideas forward, but criticism of my argument won't be as useful at this time, in part because I won't do any further work on it. This is a post I drafted in November 2023, then updated for an hour in March 2025. I don’t think I’ll ever finish it so I am just leaving it in this draft form for draft amnesty week (I know I'm late). I don’t think it is particularly well calibrated, but mainly just makes a bunch of points that I haven’t seen assembled elsewhere. Please take it as extremely low-confidence and there being a low-likelihood of this post describing these dynamics perfectly. I’ve worked at both EA charities and non-EA charities, and the EA funding landscape is unlike any other I’ve ever been in. This can be good — funders are often willing to take high-risk, high-reward bets on projects that might otherwise never get funded, and the amount of friction for getting funding is significantly lower. But, there is an orientation toward funders (and in particular staff at some major funders), that seems extremely unusual for charitable communities: a high degree of deference to their opinions. As a reference, most other charitable communities I’ve worked in have viewed funders in a much more mixed light. Engaging with them is necessary, yes, but usually funders (including large, thoughtful foundations like Open Philanthropy) are viewed as… an unaligned third party who is instrumentally useful to your organization, but whose opinions on your work should hold relatively little or no weight, given that they are a non-expert on the direct work, and often have bad ideas about how to do what you are doing. I think there are many good reasons to take funders’ perspectives seriously, and I mostly won’t cover these here. But, to
Jim Chapman
 ·  · 12m read
 · 
By Jim Chapman, Linkedin. TL;DR: In 2023, I was a 57-year-old urban planning consultant and non-profit professional with 30 years of leadership experience. After talking with my son about rationality, effective altruism, and AI risks, I decided to pursue a pivot to existential risk reduction work. The last time I had to apply for a job was in 1994. By the end of 2024, I had spent ~740 hours on courses, conferences, meetings with ~140 people, and 21 job applications. I hope that by sharing my experiences, you can gain practical insights, inspiration, and resources to navigate your career transition, especially for those who are later in their career and interested in making an impact in similar fields. I share my experience in 5 sections - sparks, take stock, start, do, meta-learnings, and next steps. [Note - as of 03/05/2025, I am still pursuing my career shift.] Sparks – 2022 During a Saturday bike ride, I admitted to my son, “No, I haven’t heard of effective altruism.” On another ride, I told him, “I'm glad you’re attending the EAGx Berkely conference." Some other time, I said, "Harry Potter and Methods of Rationality sounds interesting. I'll check it out." While playing table tennis, I asked, "What do you mean ChatGPT can't do math? No calculator? Next token prediction?" Around tax-filing time, I responded, "You really think retirement planning is out the window? That only 1 of 2 artificial intelligence futures occurs – humans flourish in a post-scarcity world or humans lose?" These conversations intrigued and concerned me. After many more conversations about rationality, EA, AI risks, and being ready for something new and more impactful, I decided to pivot my career to address my growing concerns about existential risk, particularly AI-related. I am very grateful for those conversations because without them, I am highly confident I would not have spent the last year+ doing that. Take Stock - 2023 I am very concerned about existential risk cause areas in ge
 ·  · 3m read
 · 
Written anonymously because I work in a field where there is a currently low but non-negligible and possibly high future risk of negative consequences for criticizing Trump and Trumpism. This post is an attempt to cobble together some ideas about the current situation in the United States and its impact on EA. I invite discussion on this, not only from Americans, but also those with advocacy experience in countries that are not fully liberal democracies (especially those countries where state capacity is substantial and autocratic repression occurs).  I've deleted a lot of text from this post in various drafts because I find myself getting way too in the weeds discoursing on comparative authoritarian studies, disinformation and misinformation (this is a great intro, though already somewhat outdated), and the dangers of the GOP.[1] I will note that I worry there is still a tendency to view the administration as chaotic and clumsy but retaining some degree of good faith, which strikes me as quite naive.  For the sake of brevity and focus, I will take these two things to be true, and try to hypothesize what they mean for EA. I'm not going to pretend these are ironclad truths, but I'm fairly confident in them.[2]  1. Under Donald Trump, the Republican Party (GOP) is no longer substantially committed to democracy and the rule of law. 1. The GOP will almost certainly continue to engage in measures that test the limits of constitutional rule as long as Trump is alive, and likely after he dies. 2. The Democratic Party will remain constrained by institutional and coalition factors that prevent it from behaving like the GOP. That is, absent overwhelming electoral victories in 2024 and 2026 (and beyond), the Democrats' comparatively greater commitment to rule of law and democracy will prevent systematic purging of the GOP elites responsible for democratic backsliding; while we have not crossed the Rubicon yet, it will get much worse before things get better. 2. T