The simulation argument is silly, for reasons I've previously detailed here and here. For one, the physicist Sean Carroll cleverly pointed out that the simulation argument contains a logical self-contradiction. Often, a logical self-contradiction is enough to end all discussion! Yet that's not even the end of the simulation argument's problems.
Another problem is that forecasts about future technological events within our universe don't necessarily tell us the story of the creation of our universe. Let's say we could accurately forecast that our descendants will simulate lots of universes. What does that tell us about the creation of our universe? Nothing, without further logical steps.
By analogy, let's say you plan to have a baby using IVF. Does that imply you were born thanks to IVF? No, of course not, not just by itself. Yet the simulation argument wants us to infer that if we create a universe a certain way, then our universe was also created that same way. This is an invalid inference.
But this still isn't the end of problems for the simulation argument, which just gets messier and more objectionable the more you scrutinize it. For instance, what major crimes or morally abhorrent acts might the creator of a simulation be guilty of? Torture, murder, genocide...? Wouldn't there be prohibitions against this sort of thing?
I could go on. There is just no good reason for the simulation argument to be taken seriously.
I agree with a lot of this, and generally think people should act "as if":
There is at least one of them in some 'basement' universe.
They "control all the the copies" of themself
rather than reason/act as if "I might be in a simulation". In practice I think the two don't come apart massively, but I think there are risks in just adoping this latter approach (e.g. psychological).
On the two claims I think people should follow as if they were "true":
If we reject (1) then we can't rule out the case of our observations being non-veridical[1].
This would make it hard for us to influence the 'base' reality (which for utilitarians might be where most of the potential value is).
In a large enough multiverse, (1) seems likely. (Though if (1) is false and we're in a non-veridical simulation, it's unclear how much evidence we can have about whether we're in a large multiverse[2]).
Even if (1) did not hold, we might hope that some of observations are still useful in helping us influence the basement reality e.g. mathematical/logical/ philosophical arguments [3]. Of course simulators could be be arbitrarily Cartesian-demon like, but these seem safe to me to bracket-out.
I think (2) is not too tricky to accept and widelt accepted in these parts (though I think many academic decision theories would reject). Of the range of views the axes described in this post, one can be an evidentialist/EDT-like or use the algorithmic ontology.
Regarding veridicity, I think there's some risk of circularity here (and am not certain I've thread the needle):
With non-veridical simulations, I want to say something like ruling out simulations that some smart intelligence would say are of "unreasonable counterfactuals" (or are just complete fiction - i.e. they don't aim to capture some of the same dynamics as they believe they have in their own universe/multiverse)
So for example, if the totality of what exists ("the universe") was determined by some arbitrary (to some highly intelligent observer who understood physics as much as possible ) initial conditions, them I would class the simulation that runs "the universe but with different intial conditions" as reasonable.
With respect to an intelligent observer that believed in (and had good reason to believe) in a sufficiently large multiverse, then it's plausible that many counterfactuals are "reasonable".
Conditional on being in a non-veridical simulation, beyond our prior we have an anthropic update (if we are non-updateless) but little else to inform us about the world. (Perhaps our prior has expectations over which simulations that basement beings run)
Without (1), I don't think we can necessarily rule out simulators being able to run "counterlogical simulations" (this may only seem impossible to us because of the world they are happening to simulate).
It doesn't make sense to ask questions like, Does a computer program of a mind really instantiate consciousness? That question reifies consciousness as a thing that may or may not be produced by a program. Rather, the particles constituting the computer just move—and that's it. The question of whether a given physical operation is "conscious" is not a factual dispute but a definitional one: Do we want to define consciousness as including those sorts of physical operations?
The relevant question is how much animals' experiences are discounted relative to the global poor's experiences. If they're discounted the same amount, animal welfare remains ~1000x as cost-effective [ratio between the cost-effectiveness of corporate campaigns for broiler welfare and GiveWell's top charities neglecting effects on non-target individuals].
I got this estimate based on the sentience-adjusted welfare ranges Rethink Priorities (RP) initially presented, and neglecting effects on soil invertebrates. However, I have little idea about whether chicken welfare reforms increase or decrease welfare accounting for uncertainty in welfare comparisons across species, and effects on soil invertebrates. For the sentience-adjusted welfare ranges presented in Bob Fischer's book, which are similar to the ones RP initially presented, I think effects on soil invertebrates would be much larger than those on chickens. So, trusting the values in the book, and excluding interventions decreasing uncertainty, I would focus on increasing the welfare of soil invertebrates by increasing or decreasing agricultural land. I believe this can be achieved more cost-effectively by the Good Food Institute (GFI) or Giving What We Can (GWWC) than interventions improving the conditions of farmed animals. In reality, I favour interventions decreasing uncertainty about welfare comparisons across species, and effects on soil invertebrates.
I think one of the ideas that is heavily referenced in this post is a serious infohazard, even despite being presented carefully and alongside objections. I would recommend instead that all references to this idea be removed completely.
I'll repeat the idea here only to justify my reasoning:
That AI safety researchers are likely more conscious than many of the people they interact with on a daily basis.
As an AI safety researcher, I still think this idea is very implausible, mostly because I think it implies one of several claims that are all very extraordinary from my perspective:
Either (A) I'm somehow destined to be as relevant to the origins of AGI as Jensen Huang, or (B) I'm somehow less conscious than Jensen Huang, and there's a way I could've been more conscious, or (C) everyone above an exact threshold of relevance is simulated in full detail, including obviously irrelevant experiences, which seems to undermine the efficiency premise.
(This is of course to say nothing of the simulation premise itself, which I also think is on very shaky ground.)
But even with all the hedging and counter-arguments, I can imagine this idea being intuitively compelling to some people, and leading to morally repugnant behaviour (including potentially in important decisions).
Edit: Why?
A lot of people here have a bias towards taking unusual ideas seriously / scout mindset.
As @Yarrow Bouchard 🔸 said elsewhere, strong objections aren't always forthcoming, even to shaky arguments. It took me a while to form a view on this argument myself.
Adding to Tristan's 'large enough multiverse' point, and re section iii ('We Might Get Terminated Soon'). There seem to be a few ways to think about personal continuity here. To describe two:
Continuity is causal, and termination of simulations contributes to some probability of "seeing nothing" next equal to the probability of being a simulated copy.
Continuity is content-based, and termination of simulations just means you continue to whatever remains and is sufficiently coherent to be your next moment, e.g. copies in base reality (related to Wei Dai's 'identical copy immortality').
I kinda lean toward 2 > 1. If there’s nothing special about causality that links you from one moment to the next, then what links them might just be that the later moment’s contents coherently follow on from the earlier one’s.
Continuity is ontologically tricky, but in practice we act as if there’s a fact about what we’ll experience next. Overall, I expect to see stuff even after simulations end.
(AI disclosure: After writing up my thoughts, I had Claude draft most sections of this post before rewriting them in my own words.)
If you spend too much time talking to EAs or rationalists, someone will raise the possibility that we’re living in a simulation. I’ve met people who, like Elon Musk, are pretty convinced we are, and I’ve met those who, like Joe Carlsmith, are more skeptical. Historically, the usual argument was that we’re being simulated for entertainment. As AI capabilities have increased, the argument I’ve more often heard is that the simulators are interested in AGI origins (an “AGI sim”).
You’re an EA/rationalist who’s at most one step removed from people who’ll play a pivotal role in AGI origins. If the simulators are compute constrained, you being you is best explained by a simulation rendering reality at a fidelity proportional to what the simulators care about. In other words, the vast majority of people and almost all animals might not be conscious at all.
If the simulators are compute constrained, they’ll probably just terminate the simulation once they’ve gathered the information they want. Whether or not our x-risk reduction efforts go well, we might be doomed.
I think there are many reasonable counterpoints to the above, but I personally find this account of our reality plausible, and I’ve met other EAs who feel the same way. If we took these conversational curiosities seriously, what would that imply for EA?
A. The Simulation’s Crucial Considerations
i. Why An AGI Sim?
In brief, we should think we might be in a simulation if we think advanced intelligence (perhaps our descendants) with sufficient computational resources would be likely to run many simulations of history.
Historically, the typical explanation for why they’d simulate us was entertainment. I wasn’t very convinced by this, because it just didn’t seem like our reality was optimized for entertainment. If we’re entertainment, why is history only pretty entertaining but not maximally? Do our lives seem particularly entertaining? Why is fundamental physics not simpler and more optimized for entertainment? Why the heck did they create a huge particle that does nothing but disappear in 0.3 picoseconds? Why does one of the fundamental forces care whether you’re going left or right? It just doesn’t make sense to me.
As an AGI expands its reach over its lightcone, it might want to understand what kinds of AGIs it might bump into as it expands, so it can be prepared and potentially bargain with them.[1]
A human-aligned AGI might want to run counterfactual Earths to better understand what it should value.
This would also explain why fundamental physics is as weird as it is (maybe it’s similar to that of the simulator's reality), why our world doesn’t look obviously simulated, and why you’re an EA/rationalist who’s at most one step removed from people who’ll play a pivotal role in AGI origins. There’s lots one could quibble with here, but for the purposes of this post, let’s assume we’re in an AGI sim, so we can examine which conclusions would follow.
ii. Most People And Animals Might Not Be Conscious
Especially if the simulator's reality is like ours, they might be compute constrained. Simulating a whole universe is way more expensive than rendering the sim at a fidelity proportional to what you care about. For many of us, proportional fidelity would also explain why we are who we are.
I’ve heard people treat consciousness under this hypothesis as a binary, where people are either the “chosen conscious” or not. It seems more likely to me that this hypothesis would predict people further from AGI to be rendered at lower fidelity, so they might still be conscious, just less so. Animals, irrelevant to the AGI transition, might not be conscious at all. (Maybe for the best.)
There’s a lot of pushback one could give here:
I observe myself as conscious, but instead of being Jensen Huang, I’m a quant from Boston who writes essays about chickens—not exactly the optimal allocation of simulation compute. In general, the further you are from the AGI transition, the less convincing this part of the post will be to you. If you’re an Anthropic researcher on your lunch break, you’re smirking about the author of the essay you're reading not being conscious. If you’re in Nigeria handing out malaria nets, you’re rolling your eyes.
There might be no way to cheaply approximate billions of people when those people are the AI’s training data, the demand AIs are built to satisfy, and an influence on future AI values.
More speculatively, there’s no way to know whether the simulator's reality’s physics has a notion of compute as a constraint the way our reality does. If they don’t, there’s no reason why the sim couldn't just be run at full fidelity.
Finally, history teaches us that the reasoning procedure behind the conclusion that “my in-group matters but outsiders don’t” has a dismal base rate. Atrocities like slavery, factory farming, etc remind us that the costs of getting this wrong vastly outweigh the gains.
iii. We Might Get Terminated Soon
If the simulators are computationally constrained, we should expect them to terminate the sim once they’ve learned what they wanted to know.
As always, there are counterarguments. If AGI values take time to settle, maybe the simulator will run our universe for a few million more years until the AGI converges on a stable strategy. If the simulator's reality isn’t compute constrained, maybe this universe will just keep running.
The main implications here are for longtermism. Longtermists have argued (correctly, in my view) that applying an ethical discount to the far future is philosophically dubious. An uncertain termination date implies an empirical discount to the far future. This discount's implications would be similar to those of an ethical discount.[2]
B. Implications for EA
i. Fanaticism Doesn’t Care
If you’re fully fanatical about expected value and endorse maximizing expected choiceworthiness, most of the above shouldn’t change what you do. Even if there’s an astronomically small chance we’re not in a simulation, the prospect of creating 10^50 happy lives in the far future still dominates. The exception, which I'll discuss in more detail below, is trying to influence the simulator's reality, which could be worth even more.
ii. The EA Decision Tree
If you lean towards a moral parliament, are risk averse, or just have ordinary human scope sensitivity, the situation becomes more complicated.
The two crucial considerations from section A give us a decision tree. Do the simulators render everyone at full fidelity, or at a fidelity proportional to what they care about? And do they terminate us soon, or let the sim keep running?
Full fidelity, sim terminated soon: If everyone is conscious but we'll be terminated soon, neartermism (global health and animal welfare) would likely be a better use of resources than x-risk or other longtermist work. Within neartermism, I've argued that animal welfare would be the most cost-effective use of resources.
Full fidelity, sim keeps running: Standard EA; keep doing what you've been doing.
Proportional fidelity, sim terminated soon: The global poor and animals get discounted, but the scale of animal suffering is large enough that animal welfare plausibly still looks better unless animals are hardly rendered at all.
Proportional fidelity, sim keeps running: Everything looks worse here, but neartermism is discounted more than longtermism. On the margin, this should move you toward longtermism.
iii. Proportional Fidelity's Effect on Neartermism
If animals are rendered at much lower fidelity, the scale of animal suffering could be much lower than it appears. While this would reduce the importance of animal welfare, animal welfare would plausibly still look better than global health.
The relevant question is how much animals' experiences are discounted relative to the global poor's experiences. If they're discounted the same amount, animal welfare remains ~1000x as cost-effective. However, animals probably would be discounted more than the global poor. The global poor affect the AI transition by providing AI training data and economic demand, but it's hard to come up with a convincing account of how an animal's conscious choices would materially affect the AI transition. So the scale gets less overwhelmingly in favor of animals, but unless animals are discounted up to 3 orders of magnitude more than the global poor, animal welfare still looks like a better use of resources.
Whether or not you take proportional fidelity seriously, global health and animal welfare still matter a lot in expectation. Over 800 million people live in extreme poverty, tens of billions of animals are being factory farmed at any given time, and far more suffer in the wild. There isn't a reasonable discount which stops these from being (in expectation) among the world's most important problems.
iv. Compute Fungibility
If we weren't being speculative enough, let's dial it up to 11. Assuming the simulators are compute constrained, anything that makes our sim cheaper frees up compute for other sims. If you prevent lots of wild animals from existing, that reduces the compute our world uses, and that compute gets reallocated to other worlds which have their own suffering wild animals. To generalize, if you have a good reason to think our sim is better / worse than other sims, on the margin, this would nudge us towards increasing happiness / reducing suffering.
v. Influencing The Simulator
If the simulator is human-aligned and running counterfactual Earths to understand what it should care about, then being legible about what we value could help the simulator do much greater amounts of good in their reality than is possible in our own. Alternatively, if the simulator is misaligned but is simulating us because it wants to understand what other values are out there, being legible about our values makes our simulator more likely to cooperate and do mutually beneficial trade with beings in its reality who are like us.[3] Work on shaping AI values in our reality would be one direct way to target these benefits. This would hold even if our sim will be terminated soon or is being run at proportional fidelity.
Unfortunately, there's an s-risk concern which counterbalances this. If it's easier to create suffering than happiness, or especially if you're a negative utilitarian, it's extra bad to tell a misaligned AGI what it can do to get leverage over beings like us.
vi. We Are Digital Minds
If we're in a sim and you think we matter, that's an existence proof that digital minds (ourselves) matter morally. On the margin, if you aren't already sold on the importance of digital mind welfare, the simulation hypothesis is one reason to be supportive.
C. Testing the AGI Simulation Hypothesis
There are a few ways we could get weak evidence for whether or not the AGI simulation hypothesis is true:
The AGI sim hypothesis predicts short timelines. If you're reading this post fifty years later and AGI turned out much harder to achieve than we thought and it doesn't seem likely anytime soon, that's evidence against this hypothesis.
The AGI sim hypothesis predicts there's high uncertainty about how the AGI transition will go. If our simulator is running Monte Carlo simulations, they'd concentrate the runs where there's the most variance, since it's less worthwhile to run sims where the outcome is more obvious. It is literally true that Trump being President is evidence for the simulation hypothesis. The more undetermined you think the future is, the more this should shift you towards the simulation hypothesis.
D. Your Childhood Might Not Have Happened To You
If the simulator is compute constrained, it's wasteful to run your childhood, since it isn't as relevant to the AGI transition. Instead, they might just run one sim of the world when you were growing up, save that sim's state at some checkpoint, and spin up a larger number of sims from that checkpoint state. This is similar to how you can fork an LLM conversation at the tenth turn and the forked LLM has the memory of the earlier turns even if it didn't personally experience them.
This model of the world says it's very unlikely that the instance of you who's reading this actually experienced your childhood. Even fewer instances, or maybe none at all, would have experienced humanity's earlier history.
E. So What Should We Actually Do?
If we're in an AGI sim, your childhood might not have happened to you, but the present moment where you're reading this is a real moment the simulator is watching. The value of public conversations and the importance of digital minds are two of the AGI sim hypothesis' most robust implications.
If you're an altruist and you put any credence in the AGI sim hypothesis, I encourage you to think out loud about what the credences you put in the above EA Decision Tree imply about your priorities. In a world that’s being watched, reasoning in public might be one of the most valuable things you can do.
Note: This post was crossposted from the Coefficient Giving Farm Animal Welfare Research Newsletter by the Forum team, with the author's permission. The author may not see or respond to comments on this post.
Subtitle: On vegan advocacy, effective altruism, and FIFA
I turn 40 today. Here are some hot takes.
On factory farming
1. Our biggest challenge is salience. If factory farming led the evening news, it wouldn’t last lo...
Impressions from the 21st Annual U.S. Cluster Headache Patient Conference, organized by Clusterbusters.
On the morning of the last day of the conference, E. was hanging out with other “clusterheads” outside the conference hotel when he started having a cluster headache attack. Before he could go back to his hotel room, J. offered E. his DMT v...
TL;DR:
The movement has an evidence problem. To solve it, for each intervention we need to know:
* What types of evidence that intervention needs
* What types of evidence different sources can give us
In this post, we outline a framework for evidencing an intervention through five stages:
...
The simulation argument is silly, for reasons I've previously detailed here and here. For one, the physicist Sean Carroll cleverly pointed out that the simulation argument contains a logical self-contradiction. Often, a logical self-contradiction is enough to end all discussion! Yet that's not even the end of the simulation argument's problems.
Another problem is that forecasts about future technological events within our universe don't necessarily tell us the story of the creation of our universe. Let's say we could accurately forecast that our descendants will simulate lots of universes. What does that tell us about the creation of our universe? Nothing, without further logical steps.
By analogy, let's say you plan to have a baby using IVF. Does that imply you were born thanks to IVF? No, of course not, not just by itself. Yet the simulation argument wants us to infer that if we create a universe a certain way, then our universe was also created that same way. This is an invalid inference.
But this still isn't the end of problems for the simulation argument, which just gets messier and more objectionable the more you scrutinize it. For instance, what major crimes or morally abhorrent acts might the creator of a simulation be guilty of? Torture, murder, genocide...? Wouldn't there be prohibitions against this sort of thing?
I could go on. There is just no good reason for the simulation argument to be taken seriously.