Comments
24 min read · Oct 31, 2025
Epistemic status:
This series, written for the "Essays on Longtermism" competition, was my first experiment in using Claude 4.5 Sonnet as a ghostwriter, and I learned some valuable lessons worth sharing.
While I feel relatively good about this final product, I had to do extensive editing to correct errors on Claude’s initial version, and due to serious time constraints (the original series was written in 3 days) the version I submitted for the competition contained some obvious errors I missed and, more importantly, some very subtle errors which I am quite embarrassed of.
While the large majority of the ideas in the series are my own, Claude did a good job and sometimes an excellent job of articulating them and a decent job of organizing them in a logical sequence. My main complaint is that the essays are occasionally repetitive and there are some elements which are duplicated across essays, however partly this was intentional in order to make it so that each essay could easily be read as a fully stand-alone piece. Claude can also be wildly mis-calibrated, often claiming something is “the best” or “the most important,” when there is no strong evidence for this or consideration of alternatives.
Additionally, while I have tried to remove any subtle errors, I have occasionally realized I missed one that was actually quite important, and so I apologize if any remain which I have missed. This is a big challenge, as Claude can sound very eloquent at times and yet be completely wrong, misconstruing ideas in logical sounding yet subtly misleading ways.
My biggest recommendation to anyone using AI for ghostwriting is to very, very carefully read the writing and think about it and make sure the ideas are correct, as well as to perhaps explicitly mention AI has been used in the writing and there may be subtle errors of this type.
With those caveats in mind, here’s a link to the initial version of the essay submitted to the “Essays on Longtermism” competition.
I also want to note that while this series represents some of my best ideas over the last few years, it does not have the level of finesse and flavor of posts I typically like to write, perhaps with the exception of “Viatopia and Buy-In” which was an excerpt from my in-progress essay on “Deep Reflection.”
Deep Reflection is the comprehensive examination of crucial considerations to determine a strategy to achieve best achievable future; this series was largely based on my research on Deep Reflection. (Deep Reflection Summary Available Here)
As mentioned, this essay, easily readable as a stand-alone piece, is the second essay and part of my submission to the "Essays on Longtermism" competition.
Brief and Comprehensive Series Explainers Here
TL;DR
This essay provides the theoretical foundation for why viatopia matters. Viatopia, a concept introduced by Will MacAskill, refers to “a state of the world where society can guide itself towards near-best outcomes.”
This essay establishes the multiplicative crucial considerations framework: dozens to over a hundred considerations (normative, epistemic, strategic, empirical) interact multiplicatively in expectation to determine future value, meaning comprehensive reflection may achieve orders of magnitude greater expected value than addressing considerations individually.
It introduces the instrumental commoditization thesis: as AI capabilities grow, implementation becomes trivial, while determining good directions becomes everything, making values and institutional design work high-leverage pre-AGI Better Futures interventions.
Will MacAskill's bootstrapping mechanism shows how researching viatopia reveals crucial considerations that justify continued reflection, creating a self-reinforcing loop that helps prevent premature lock-in. The diversity mechanism extends this by ensuring multiple well-developed viatopia paths exist in advance, creating an initial incentive to pause and carefully compare alternatives rather than defaulting to whatever option is most readily available.
One key advantage of viatopia, explored more extensively in the next essay, lies in creating tractability through buy-in. It’s politically feasible, whereas implementing comprehensive reflection (another term for “Deep Reflection”) directly may not be.
The essay explores parallels and differences between MacAskill's Better Futures framework and my Deep Reflection work, explains why early strategy research is uniquely high-leverage, and discusses the fundamental design challenge of maximizing both human agency and future value simultaneously. This theoretical foundation explains why the concrete mechanisms in subsequent essays matter.
MacAskill's Better Futures and Viatopia as the Unifying Framework
When Will MacAskill shared his Better Futures essay series with me after I requested feedback on my own work, one concept stood out as perhaps the single most important contribution to longtermist strategy: viatopia. While MacAskill discusses numerous interventions across his series, I believe viatopia serves as the unifying framework that makes all other Better Futures work coherent. Without viatopia, we're simply listing isolated interventions; with it, we have a strategic vision for how to navigate from our current world to one capable of achieving the best possible future.
MacAskill states: "Plausibly, viatopia is a state of society where existential risk is very low, where many different moral points of view can flourish, where many possible futures are still open to us, and where major decisions are made via thoughtful, reflective processes."
This concept is not merely theoretical for MacAskill. He has indicated that viatopia is one of the most important ideas in his Better Futures series and that Forethought, the strategy research organization, should possibly consider working on it. Interestingly, while he emphasized its importance, he devoted relatively little space to it within the series itself, noting his intention to explore it further at a later date. His only published piece specifically on viatopia focuses on "bootstrapping to viatopia," the mechanism by which we might progressively delay major civilizational decisions as we identify more crucial considerations worth reflecting on, such as by repeatedly extending a global constitutional convention as we realize there are more factors we should carefully consider.
The Instrumental Commoditization Thesis: Why Direction Becomes Everything
One important consideration for understanding why viatopia work is so urgent is what I call the instrumental commoditization thesis: as AI capabilities grow exponentially, implementation becomes increasingly trivial while direction becomes overwhelmingly important. Very soon, advanced AI will enable us to implement nearly any institutional design, technological development, or societal intervention we can specify clearly. The binding constraint shifts from "Can we do this?" to "Should we do this? What exactly should we do?"
This creates a critical dynamic: whatever institutional designs and value frameworks are well-developed and readily available when this transformative capability arrives may be what we actually implement. When faced with unprecedented power and pressure to make decisions about humanity's trajectory, we may use the tools lying around rather than conducting an exhaustive search through all possibilities. This is simply because having vetted, concrete proposals available makes them vastly more likely to be adopted than proposals that exist only as vague ideas requiring years of development.
This means that focusing attention on determining the best possible goals, developing multiple viable paths to good futures, and creating the deliberative infrastructure to choose wisely among them may represent high-leverage Better Futures work in the pre-AGI period. In other words: we should be skating to where the puck is going. Knowing what the best possible shape of the universe could be, or at least knowing the best process for determining this, becomes the final challenge once AI can handle everything else. Values and strategic direction may be the final key areas of human advantage, which we either can’t or don’t want to automate away with AI, making them high-leverage focus areas for human effort before transformative AI arrives.
Context from Deep Reflection: The Broader Framework
This essay is based on my research on "Deep Reflection," an in-progress 35,000-word essay exploring the pre-paradigmatic field of comprehensive reflection across all crucial considerations (for example, what we might hope for from a “Long Reflection”). This research aims to ensure humanity achieves not only existential security but the best achievable future.
While I had been working various forms of utopia research from long before I wrote my draft of “Ways to Save The World”/“Paths to Utopia” in 2021, I originally started this Deep Reflection research in March 2025 for the Existential Choices Debate Week, which I had requested in July 2024, based on Will MacAskill’s 2022 book “What We Owe The Future.”. An intermediate summary version of Deep Reflection is available here for those interested in the full theoretical framework.
Because I cannot finish the entire Deep Reflection essay series by the competition deadline, I am summarizing the most important points of it here to explain why Viatopia is important and how it connects to Deep Reflection.
The central concept of Deep Reflection is straightforward: a comprehensive examination of all crucial considerations to determine how to achieve the best possible future, followed by implementing the results of that reflection. This is the broader class of interventions that includes proposals like Will MacAskill and Toby Ord’s "Long Reflection", but encompasses any process of sufficiently thorough and comprehensive reflection, whether implemented as a single extended period or as an ongoing process of careful deliberation at each step.
The full Deep Reflection essay establishes several key points that provide essential context for understanding why viatopia matters:
Path-dependence and lock-in risks: As technology advances rapidly, humanity may unknowingly take paths that create path-dependence, increasing probability of eventual lock-in. By path-dependence, I mean that we take paths which are naturally highly unlikely to be reversed. Path-dependence may also come in milder forms where lock-in of certain outcomes becomes slightly more likely, yet such path-dependence could still have large impacts on expected value. MacAskill's Better Futures series explores multiple mechanisms by which persistent path-dependence becomes more probable after transformative AI. Given the vast time scales of potential futures, it seems quite possible that humanity will eventually fall into some stable, locked-in attractor state. The question is whether we deliberately choose that state through careful reflection or blindly stumble into it. (Note that while eventual lock-in seems quite likely, we may choose to “lock in” to a state that includes significant or even maximum personal choice, a sort of “locked-in option value” allowing ongoing freedom of action.)
The urgency of early strategy work: We face a potentially fleeting window of opportunity. Currently, relatively few people are trying to influence how advanced technology is used or what we do with the deep future, but this will soon change dramatically. Once transformative AI arrives, many actors will push for their preferred trajectories, and coordination becomes vastly more complex. The institutions and ideas we have ready at that moment may have outsized influence simply through availability.
Multiplicative Crucial Considerations: Why Comprehensive Reflection Matters
To understand why viatopia is essential, we must first understand the multiplicative crucial considerations framework, which forms the theoretical foundation for why comprehensive reflection has such enormous expected value.
A crucial consideration (Nick Bostrom’s term), is any factor that could materially influence the value of the future, which we must understand in order to act correctly. Importantly, this extends far beyond normative questions about what "the good" ultimately consists of. Crucial considerations include:
The crucial insight is that these considerations seem to interact multiplicatively in expectation, rather than additively. If we get one major consideration seriously wrong, it doesn't just reduce the value of the future by some additive amount; it could reduce it by a multiplicative factor. Get several wrong, and the multiplied reductions compound. With potentially dozens to over a hundred such considerations, with some considerations potentially affecting value by factors of 2X, 10X, or even more, the difference between getting all of them right versus even getting ‘merely’ most of them right could represent many orders of magnitude in expected value.
Moreover, it is extremely difficult to predict complex events decades out, to say nothing of the impact of our actions on events in the deep future. When we combine this with the fact that there are numerous conceptually and strategically interacting crucial considerations, we get a state of 'complex cluelessness,' in which trying to impact just one of these crucial considerations in a vacuum may have catastrophic flow through effects on other crucial considerations.
This creates a fundamental challenge for narrow trajectory change interventions. If we focus on changing one aspect of the future (say, ensuring certain governance structures or advancing certain technologies), we might succeed at that narrow goal while inadvertently creating negative effects on other crucial considerations we haven't thought carefully about. The interactions are complex enough that we can't simply work on considerations one by one and expect to achieve good results.
This framework explains why comprehensive reflection processes like Deep Reflection and early versions of it enabled by Macrostrategy Research Automation (discussed more in “Shortlist of Viatopia Interventions”) have such enormous potential value. Rather than trying to address crucial considerations one by one while remaining clueless about their interactions, comprehensive reflection systematically examines all considerations together, understanding how they relate and ensuring we don't inadvertently undermine progress on one front while advancing another. The expected value difference is not marginal - it represents orders of magnitude in expectation, despite the admittedly significant challenges in tractability.
In “No Easy Eutopia” Fin Moorhouse and Will MacAskill have articulated a related but subtly different concept they call ‘multiplicative factors’. Their framework focuses specifically on qualities that the future itself must have to achieve high value. They argue these factors interact multiplicatively such that getting them all right may be essential to achieve near-best futures. This is an important insight, and my crucial considerations framework includes such factors as a central component.
However, my framework extends more broadly to include epistemic, strategic, and empirical considerations that affect our ability to determine and achieve good futures, not just the normative qualities those futures must possess. This distinction matters because it suggests we face an even more complex challenge than identifying the right end-state properties: we must also navigate the strategic and epistemic challenges of reliably getting there.
I believe both frameworks point toward similar conclusions about the importance of comprehensive reflection, though they emphasize different aspects of why such reflection is necessary.
The fragility of value
Carl Shulman raised an important critique of my crucial considerations framework, and as I’m unsure I can do full justice I will link to his original (now retracted) comment (on MacAskill’s series) for clarity.
My understanding is that he is arguing that if there truly are dozens to hundreds of factors we need to get right, and they interact multiplicatively, then we're likely almost certainly doomed anyway, we couldn't possibly address them all, so we'd lose most value regardless.
He suggested we should instead focus on ‘non-fragile’ worlds and getting the main, broad considerations right, such as preventing autocracy, because we are much more likely to achieve near maximum value in worlds where these conditions are sufficient. MacAskill responded to Shulman suggesting there may be only a small number of factors affecting future value, which would make targeted interventions more promising. MacAskill also responded with further considerations I won’t elaborate on here, again original comment linked above.
I believe the answer to the critique lies hidden within it: Shulman points out that non-fragile worlds likely have enhanced epistemics. I agree; in particular, I think that automated macrostrategy research can dramatically increase our capacity to analyze these considerations systematically, allowing us to get sufficiently good answers so as to create relatively robust interventions. These interventions may act on crucial, broad systemic factors, and include self-reinforcing feedback loops, which get us into progressively better strategic positions, allowing us to move progressively closer to Better Futures.
This is precisely why early work on automating strategy research is so valuable, it scales our ability to handle complexity. MacAskill has made public statements indicating he is a big advocate of automated macrostrategy, indicating he may agree with this, but just isn't stating it here explicitly.
Furthermore, even without advanced automated research tools I believe that we may be able to find at least a few good proxies for progress toward better futures, just as we can find high leverage AI safety interventions which create progress toward preventing existential risk. The strategic cluelessness problem is real, but it's solvable through sufficiently sophisticated analytical processes, especially those leveraging advanced AI.
Early Strategy Research: A High-Leverage Window of Opportunity
Right now, we need intensive work on strategy and interventions to navigate the intelligence explosion successfully. We must develop early macrostrategy research tools which take advantage of increasing compute as AI progresses, and we must develop key community infrastructure and viatopian mechanisms that can be implemented early and move us into progressively better strategic positions over time. This work should ideally happen before transformative AI arrives because path-dependencies may begin accelerating once such capabilities exist.
Furthermore, there is currently unusually high leverage in framing the debate around what we should do with transformative AI once it arises, as currently there are relatively few people trying to influence the debate, but this will soon change.
As MacAskill points out, another factor to consider is that certain types of work such as field-building work and building critical infrastructure are especially important early on in order to make an intractable seeming area become tractable. Additionally, these types of ‘meta’ interventions have effects which compound over time, making early work especially high leverage.
Additionally, as pointed out by Lizka Vaintrob and Owen Cotton-Barratt, we are in a crucial moment where we can do the preparatory work necessary to build the most important AI tools as early as feasible. With the speed of AI progress, even a small advancement in the time at which we get certain critical applications could be decisive, and certain crucial viatopian applications might not be provided by the market at all.
Once sufficiently advanced AI exists, it will generate strategy better than humans can. At that point, research tools developed with effective scaling in mind may still continue to have an advantage, however there will likely be a few dominant factors which will be the primary determinants of what we do next. These are the kinds of factors we need strategy and interventions to target in advance. Some important factors include:
It is the work we do now which could determine much of this strategic position in the future. By doing the foundational strategy work now, we can be well-positioned when AI capabilities advance; we can strategically work to shape these systemic factors, which will in turn become the primary determinants of whether humanity realizes or squanders our vast potential.
We must get in a good position for having effective values reflection now, before multiple groups are trying to influence values and strategic direction. Once transformative AI approaches, numerous actors with different value systems will push their preferred trajectories. The actors who have done the most thorough preparatory work on institutional designs and value reflection processes will be disproportionately influential. This makes comprehensive processes for broad societal research, reflection, debate, and experimentation around values increasingly urgent. We need them ready before path-dependencies accelerate.
Why Values Matter Most: Beckstead's Insight
An essential insight, articulated early on by Nick Beckstead, is that if we have the right values, we can always figure out how to achieve them, but if we have the wrong values, we might go farther and farther down the wrong path until any impetus to change our values back has been permanently extinguished.
This suggests a crucial prioritization: the trajectory of moral and epistemic progress may be especially essential. This is one of the main reasons (another being x-risk via the vulnerable world hypothesis, as articulated by Bostrom) some technologies need to be slowed down or avoided while others should be differentially accelerated. It's also why upgrading social institutions to more effectively promote moral reflection and values learning deserves focus. Systems like political institutions, economic structures, education systems, parenting and childcare approaches, and mental health support all influence how humans think about values and make moral progress over time.
The centrality of values also explains why systematic value reflection infrastructure (institutional and technological infrastructure for helping humans reflect on values, experiment with different frameworks, engage in substantive debate, and systematically evolve toward better values) may be among the highest-leverage interventions. This will become even more true as AI capabilities grow. When implementation becomes easy, having the right goals becomes everything.
Viatopia and Buy-In
One important aspect of viatopia (explored extensively in the next essay) is that viatopia has the potential to create tractability through buy-in. While implementing comprehensive reflection directly might seem infeasible or politically unpalatable, viatopia represents an intermediate state that is potentially more politically feasible. It doesn't require everyone to become explicit longtermists or to make all decisions based on far-future considerations. Instead, it can potentially include structures that naturally lead toward better reflection and decision-making, while simultaneously remaining appealing to diverse stakeholders with different values and priorities.
Convergence and Divergence with MacAskill's Framework
Given the overlap between MacAskill's Better Futures work and my own Deep Reflection essay, it's worth explicitly noting where our thinking converges and diverges. We arrived at surprisingly similar conclusions through somewhat different paths, and understanding both the similarities and differences helps clarify the overall framework.
Convergences between our frameworks:
Both MacAskill and I independently developed viatopia-like concepts as central to our thinking about the far future. His term "viatopia" appears in his Better Futures series; I used "Seed Reflection" as my term in 2025 when writing my Deep Reflection essay, and earlier used "Paths to Utopia" as a working title for a book draft in 2021 (before discovering the EA community) exploring mechanisms for navigating from the current world to ideal futures.
We both emphasize the concept of existential compromise (myself) or grand bargain (MacAskill) - the idea that with sufficient resources (likely enabled by advanced AI), we can structure arrangements that are massively positive-sum, satisfying diverse stakeholders while still making progress toward better futures. In my case, "existential compromise" was inspired by Nick Bostrom and Carl Shulman’s suggestion of a possible compromise between super-beneficiaries and other beings.
Both of us heavily emphasize path-dependence and the mechanisms by which early decisions can have persistent effects on trajectories. MacAskill's Better Futures series explores multiple ways persistent path-dependence becomes more likely with transformative AI. My work similarly stresses how transformative AI creates both opportunities and risks for path-dependence leading to eventual lock-in.
Key differences and my evolution:
One of the most important concepts I learned from MacAskill's work is "doing the good de dicto"—being motivated to pursue fundamental good rather than specific goals. This was a major takeaway from his Better Futures series that I had not articulated in my own work. The idea is that some actors are motivated to promote whatever turns out to be good fundamentally (the good ‘de dicto’), rather than being attached to specific outcomes they currently believe are good (the good ‘de re’). For such actors, if they learned that something different was actually better, they would change their behavior accordingly. This attitude of pursuing the good fundamentally, whatever it consists in, is essential for reliably achieving great futures.
On cooperation versus strategic advantage: my initial approach emphasized realpolitik considerations and the importance of influencing whichever party achieves a decisive strategic advantage. MacAskill's work places greater emphasis on cooperation and positive-sum coordination. Through his influence and that of Forethought Foundation's research, I've evolved toward placing cooperation more centrally in my framework, while still recognizing that considerations of strategic advantage remain relevant in some scenarios.
On crucial considerations versus multiplicative factors: As explained earlier, these frameworks are related but distinct. Both recognize multiplicative interactions between factors affecting future value, but my crucial considerations framework encompasses a broader range of epistemic and strategic factors, while his multiplicative factors focus more specifically on properties the future must have to achieve high value.
Bootstrapping to Viatopia & The Diversity Mechanism: Creating Deliberation Feedback Loops & Natural Incentives for Reflection
One of the most important mechanisms for achieving viatopia lies in what Will MacAskill calls "Bootstrapping to Viatopia," in his only published piece specifically on the concept. The core insight is that researching viatopia itself reveals crucial considerations that justify continued reflection, creating a self-reinforcing loop. As we investigate what makes futures good and how to achieve them, we discover more factors worth thinking through carefully before making major civilizational commitments. This creates compelling reasons to delay irreversible decisions, which enables more research, which reveals even more important considerations.
MacAskill illustrates this with the example of a global constitutional convention that gets repeatedly extended. Initially, stakeholders might plan to deliberate for some fixed period, then implement their conclusions. But as they research, they realize there are additional crucial considerations they hadn't adequately examined—perhaps about the moral status of digital minds, or about institutional designs for value evolution, or about the long-term effects of different technological development sequences. These discoveries create legitimate reasons to extend the convention rather than prematurely committing. The process of developing viatopia proposals generates justification for continuing to develop them rather than hastily choosing among incomplete options.
I have been thinking about a "diversity mechanism" which could complement and extend this bootstrapping mechanism. While MacAskill's mechanism creates pauses by revealing more considerations worth examining, the diversity mechanism helps kick off this bootstrapping mechanism by developing multiple attractive alternatives in advance. The core mechanism works as follows: if only one or two viatopia proposals are well-developed when the ability to reliably shape the future becomes available (likely through transformative AI), decision-makers might simply implement the available option without extensive deliberation. After all, if there's only one concrete plan developed, using it may seem obvious. But if dozens of compelling viatopia designs exist, each well-developed and attractive in different ways, this creates a natural forcing function for serious comparative evaluation. The existence of multiple good options makes it obvious that we should pause and carefully consider which path is best, rather than defaulting to whatever happens to be most prominent.
This diversity mechanism can be implemented in advance by deliberately developing multiple well-specified visions for viatopia now, before transformative capabilities arrive. This differs slightly from MacAskill's bootstrapping, which emphasizes using temporary commitments that can be extended as global conditions evolve. The diversity approach front-loads the development of alternatives, ensuring that when crucial decisions must be made, we face a rich menu of thoughtfully developed options rather than a simple binary choice or an obvious default. Both mechanisms ultimately serve the same goal, preventing premature path-dependence toward suboptimal futures, but through complementary pathways that can reinforce each other. The more viatopia proposals we develop in advance (diversity mechanism), the more considerations we're likely to discover requiring iterative extensions when it comes time to decide (bootstrapping mechanism), which justifies both further research and maintenance of option value as we converge toward wiser choices.
Maximizing Agency AND Value: A Core Design Constraint
One of the most fundamental design challenges for viatopia is that we want to maximize both human agency and the value of the future - and these often trade off against each other.
The tension arises because humans sometimes go down paths toward attractor states, which make exploring other paths increasingly unlikely, even if those alternative paths would have been better from their all-things-considered perspective if they had the chance to experience both paths and then decide which they preferred. Once committed to certain trajectories, whether through technology, institutions, or value evolution, reversing course becomes increasingly difficult. Yet if we restrict people's freedom to choose certain paths and force them to reflect in order to choose a better path, at least in a sense, we've thereby reduced their agency.
This means viatopia must be designed to score as high as possible on both dimensions simultaneously. We need structures that allow maximal human freedom and choice while also systematically enabling effective exploration of the values and possibilities space. This is a central design problem.
Examples of just a few other tradeoffs which we may want to systematically explore when designing viatopia mechanisms:
Understanding these tradeoffs and design factors helps flesh out the whole space of viatopia intervention possibilities; essentially creating a "tech tree" for viatopia mechanism design where we map out different approaches and their implications. This systematic exploration of design space is itself important work that can help identify which mechanisms might best balance agency-value and other trade-offs.
One reason for developing human-centric versions of viatopia, such as the Children's Movement I explore in a subsequent essay, is that we may want to keep humans as the main factor determining values into the future, allowing value evolution over long timescales relative to the speed of AI progress, at least until we're confident about allowing AI significant influence over value evolution. Different viatopia designs make different choices about this crucial question.
Mechanisms for Aggregating and Improving Values
A related design challenge is that people hold different values, and we need systems for effectively aggregating this diverse information while helping ensure the best values emerge through debate, research, concrete testing, and informed decision-making, all while guarding against premature path-dependence.
Importantly, the fact that people have different values is not merely a problem to be solved but a feature that can be leveraged. Value diversity allows market-like mechanisms to function effectively. It provides a wide space to draw from, with different people holding different perspectives and having different incentives to work on various aspects of the overall value question. This distribution of attention and effort across the values landscape helps us explore more thoroughly than any single perspective could achieve.
Different viatopia designs take different approaches to value aggregation and improvement:
One possible key to this may be to create a global metatopia, echoing Atomic Communitarianism (Alexander) or “meta” option (Karnofsky): we could have multiple Viatopian approaches running in parallel, whether geographically or in simulation, learning from each other, allowing maximal agency in designing and choosing living conditions for individuals and communities, and allowing for evolution over time as we discover which mechanisms work best in practice.
"Keep the Future Human": An Important Consideration
One framework worth considering is what Anthony Aguirre calls "Keep The Future Human." While Aguirre’s main rationale for keeping the future human is ensuring AI is controllable, I believe another important reason for keeping the future human is that we may not want AI to hyper-optimize everything for us immediately, but rather prefer allowing human values to evolve carefully over time in societies that aren't radically different from our current experience.
This perspective provides part of the justification for slower, more human-centric viatopia paths. Rather than rushing to implement whatever current AI systems suggest might be optimal, we might prefer trajectories where humans maintain primary agency over value determination, at least for an extended period. This allows values to evolve in ways humans find meaningful and can understand, rather than accepting optimization according to values we haven't fully reflected on. Bostrom explores similar ideas (among many, many other ideas) in his book “Deep Utopia.”
This doesn't mean rejecting AI assistance entirely. Rather, it suggests a paradigm of human-AI symbiosis (discussed further in “Shortlist of Viatopia Interventions”) where “tool AI” or “oracle AI” augments human capacity for reflection and exploration without displacing humans as the primary agents. This bidirectional feedback loop where humans give AI context while AI helps humans improve themselves may be essential from a strategic perspective for maintaining meaningful human relevance and agency as AI capabilities grow.
Different viatopia designs reflect different positions on this question. Some might emphasize rapid AI-enabled optimization once we've achieved sufficient clarity on values. Others might prioritize maintaining human-like conditions and slower evolution even if it means taking longer to reach optimal states. Yudkowsky's (Renounced) “Coherent, Extrapolated Volition,” in sharp contrast to his "Fun Theory" demonstrate the two extremes.
The diversity of approaches allows different philosophical positions to coexist and compete in the marketplace of ideas, though it is essential there is overall coordination between the different viatopia options if we should decide to pursue Metatopia, so that no viatopia ends up destroying another, again Alexander’s Atomic Communitarianism is relevant here.
Conclusion
The preceding context establishes a few of Viatopia's core features and why viatopia is essential: it serves as the practical mechanism that makes Deep Reflection and ideal futures achievable. But understanding that viatopia is important doesn't tell us how to actually create it. That requires concrete analysis of the motives and available actions of key stakeholders.
In the next essay, "Viatopia and Buy-In," I perform concrete stakeholder mapping to identify practical pathways for achieving viatopia. While this essay established the theoretical case for why viatopia matters, the next essay addresses the challenge of actually creating it by analyzing what AI labs, governments, and the general public can each do given their different incentives, capabilities, and constraints. It demonstrates how viatopia can be framed to appeal to diverse stakeholders' interests and shows that viatopia is not just theoretically desirable but practically achievable through existential compromise. This stakeholder analysis bridges from theoretical arguments to practical implementation pathways.
