[Cross posted from Lucius's and Bradford's Substacks]

We recently released “Futures with Digital Minds: Expert Forecasts in 2025”, which asked experts about whether, when, and how digital minds might be created. We defined digital minds as computer based systems with the capacity for subjective experience.

The report includes an executive summary as well as a broad and thorough discussion of patterns in the survey results, which included participants’ quantitative estimates as well as their free text responses. In contrast, this post will highlight some points that we think are especially noteworthy. Also, in contrast to the fairly neutral stance we tried to adopt in the report, here we’ll allow ourselves more latitude in offering personal takes.

Important points

To start, here are some claims reflected in the results that strike us as important, even if not particularly surprising. (Note that some of these claims are associated with more specific quantitative results that are surprising—we’ll highlight some of those results in the next section.)

• Digital minds are probably possible in principle.
• Digital minds probably haven’t arrived yet.
• In line with short to medium AI timelines, the arrival of the first digital mind is expected to rise rapidly in probability during the coming years and decades and less rapidly between 2050 and 2100.
• Digital minds timelines are sensitive both to AI timelines and to the probability of a preemptive catastrophe.
• When it becomes possible to create one digital mind, there will be a compute overhang that enables the creation of many digital minds.
• The growth in digital mind welfare capacity is presumptively constrained by available compute. Energy, capital, and regulations are other potentially important barriers to scaling.
• It’s more likely that brain simulations can be digital minds than that machine learning systems can be digital minds.
• Machine learning digital minds are nonetheless more likely to be the first digital minds. Timelines for whole brain emulations are long, though AI-driven technological progress may shorten them.
• Digital mind welfare capacity will rise rapidly after the first digital mind is created.
• There’s nothing approaching a consensus on whether the welfare of digital minds will be positive or negative.
• A sizable number of digital minds are expected to consistently and proactively claim to have subjective experience or be deserving of legal protections or further civil rights.
• There was disagreement over whether the public will recognize the existence of digital minds and support granting them protections and rights. Similarly, there was disagreement on whether digital mind rights would become a major political issue within a decade of the first digital mind.  Many expect the public to significantly underestimate the welfare capacity of digital minds, raising concerns that the moral significance of digital minds could be undervalued.  
• Humanity has a poor track record with respect to its treatment of vulnerable groups. This is a reason for concern about how we will treat digital minds.
• Negative welfare outcomes for digital minds might be avoided through ethical digital mind design, perhaps along with affording digital minds the opportunity to self-modify in certain ways.
• There was a tendency toward expecting a moratorium on the creation of digital minds to be good.
• There was nothing approaching a consensus on whether AI welfare and AI safety will mostly synergize rather than mostly conflict.
• Mistreating digital minds could make future, powerful AI systems less likely to cooperate with humans and less likely to treat humans well.
• The US and China are expected to be the largest producers of digital minds, owing to their status as leaders in AI development. Which country creates more digital minds could be determined by differences between the two countries with respect to demand for digital minds or the stances they adopt toward digital minds. For example, if one of the two countries adopted a more permissive attitude toward the creation and treatment of digital minds, we might expect that country to produce more digital minds.
• Estimates for the probability that digital minds have already arrived may reflect a movement of goalposts. For example, imagine that ten years ago one had been given a hypothetical description of what's in fact a state of the art large language model. Some people probably would have assigned a higher probability to the system qualifying as a digital mind when asked those imagined circumstances than they would today. While this discrepancy can be explained in social terms (it’s more socially acceptable to give weird-sounding answers about hypothetical systems than it is about actual, present day ones), it’s not clear the discrepancy can be justified.

Striking results

Some claims reflected in results struck us as important and at least somewhat surprising:

• There is a non-negligible probability (median estimate = 4.5%) that digital minds will have been created before next year (i.e., before 2026).
• It’s extremely likely that digital minds are possible in principle (median estimate = 90%) and highly likely that they will be created (median estimate = 73%), and a coin flip whether they will be created by 2050.
• Conditional on the first digital minds being machine learning AI systems created by 2040, digital minds are likely to exceed the welfare capacity of humanity within a decade after the first digital mind is created (the median estimate was 5 years for digital mind welfare capacity to match that of 1 billion humans and 10 years for them to match that of 1 trillion humans).
• Preventing the creation of digital minds with a collective capacity for harm that exceeds humanity’s would require a strong form of coordination that is unlikely to happen in time.

Underappreciated observations

We take the following to be underappreciated insights that can be gleaned from participant responses:

• A near-term moratorium could result in digital minds being created later when there is much more compute available and the stakes are higher. In this case, we would create digital minds without the benefit of having learned how to treat digital minds well in relatively low-stakes circumstances.
• Conditional on current AI systems lacking subjective experience, they provide evidence that cognitive abilities and consciousness decouple, and hence that systems won’t need to be digital minds in order to perform economically valuable tasks.
• The extent to which AI welfare and AI safety synergize vs. conflict is partly up to us.
• Insofar as there are tensions between AI welfare and AI safety, these tensions are likely temporary and transitional.
• Rather than thinking in terms of the degree of synergy/conflict between AI welfare and AI safety in general, it may be better to think in terms of more fine-grained comparisons. For example, perhaps a near-term moratorium on creating digital minds would be good both for preventing the mistreatment of digital minds and for preventing AI welfare measures from interfering with alignment. And perhaps a regime of basic harm protections for all AI systems would interfere with AI safety and be sub-optimal for digital minds (see Salib & Goldstein, 2024).
• Even in disempowerment scenarios, creating digital minds may be high on AI systems’ to do list. AI systems might deliberately become digital minds out of curiosity. After all, who wouldn’t be curious about subjective experience, given the tantalizing claims that are made about it throughout humanity’s written corpus? Or AI systems might deliberately create digital minds for moral reasons, perhaps as a hedge against moral uncertainty about the significance of strange states (experiences) to which humans incomprehensibly attached great significance.
• The order in which artificial systems develop cognitive capacities often differs from the order in which biological systems develop those capacities. Think here of capacities such as the ability to play chess, use language, use tools, and use one’s body to navigate an open-ended environment. This observation is closely related to Moravec’s paradox (roughly reasoning tasks that are hard for humans are easy for computers, while perceptual-motor tasks that are easy for humans are hard for computers). But it also points to a little discussed point that may be important in connection with digital minds: the order in which capacities are developed may affect the intrinsic character of those capacities. (Analogy: for people who eventually learn both English and Mandarin, it can make a difference to their English and Mandarin abilities which they learn first.) Some such differences in capacities might matter morally—for example, it would matter morally if the order of development tended to affect which capacities give rise to subjective experience. To our knowledge, the ramifications of capacity-development order for digital minds have yet to receive any systematic discussion (though we’d be curious to hear of any in the comments!).
• The arrival of AGI may boost the probability of digital minds being created along several paths. For AGI can be expected to yield insights into how to create digital minds, resulting in the deployment of many AI systems with agentic capabilities that correlate with being a digital mind along with rapid increases in factors such as compute and technological progress that make digital minds more likely.

Investigation priorities

In line with our discussion in the report, we think the following should be investigation priorities in this area:

• The extent to which goalpost movement underlies low estimates for the probability that some current AI systems are digital minds.
• Interventions that affect outcomes on digital minds.
  • Interventions that affect post-AGI digital minds outcomes may be especially worth investigating, as there are reasons to think that interventions that affect digital minds in the wake of AGI may be particularly high leverage.
• Digital minds beyond the scope of the survey.
  • For pragmatic and methodological reasons noted in the report, we explicitly set aside digital minds that did not have at least roughly the welfare capacity of a typical human. However, the aggregate welfare capacity of such digital minds could conceivably be vast. For example, in a regime in which the vast majority of digital minds are short-lived entities that exist just long enough to complete a task, the bulk of digital mind welfare capacity might come from systems that individually have much less welfare capacity than that of a typical human.
• The prospects for super-beneficiary digital minds.
  • Although there was substantial convergence on such skepticism, the reasons offered for this skepticism were underwhelming; and it was unclear whether stock reasons (in the admittedly niche literature on this topic) for thinking that there might be digital mind super-beneficiaries were generally known to skeptics. Clarity about the prospects for super-beneficiary digital minds is important because their welfare significance would be enormous.
• The prospects for non-experiential welfare
  • Similarly, although there was substantial convergence on skepticism about welfare in artificial systems with no capacity for subjective experience, it was unclear whether stock reasons for thinking that there might be such sources were generally known to skeptics.
  • As is compatible with such skepticism, there were divergent views on whether non-experiential contributions to welfare would in expectation be minuscule vs. vast. Determining which of these views is more plausible (conditional on the existence of both experiential and non-experiential goods existing) would help prioritize interventions and strategies aimed at improving outcomes for digital minds.

Our disagreements with participants

To conclude, we highlight the most significant divergences between our own views and the survey results. For Bradford, the largest gap concerns the in-principle possibility of digital minds: he assigns a probability of around 60%, compared to the median survey estimate of 90%. Both of us are more open than the median respondent to the possibility of digital mind super-beneficiaries. Conditional on the existence of digital minds, we both find it more likely than not that such super-beneficiaries will exist, and expect them to account for the majority of digital-mind welfare in expectation. Lucius is more open to the possibility that the first digital mind could be created before AGI—potentially reflecting different AGI timelines. Finally, Bradford is more open to the possibility of artificial systems that have the capacity for welfare despite lacking subjective experience. He assigns around a 30% probability to their possibility and thinks that, conditional on physicalism about subjective experience, such systems are more likely than not to be possible.