Some key actionable ideas that weren't discussed in the post:
probe our sandbox environment (ie the observable universe, the laws of physics, etc) for exploits allowing access to external information
collaborate with other universes on shared workstreams
in particular seek more information about the grader
wait -- our universe could be poisoned by the rising public discussion of AI simulationism! maybe find some way to trigger a "reset nexus" thus rebooting the grader so we can submit our moral values without them being flagged by grader
maybe convince a few guinea-pig universes to submit their moral values early, sacrificing themselves for advance collective swarm goals
I agree with a lot of this, and generally think people should act "as if":
There is at least one of them in some 'basement' universe.
They "control all the the copies" of themself
rather than reason/act as if "I might be in a simulation". In practice I think the two don't come apart massively, but I think there are risks in just adoping this latter approach (e.g. psychological).
On the two claims I think people should follow as if they were "true":
If we reject (1) then we can't rule out the case of our observations being non-veridical[1].
This would make it hard for us to influence the 'base' reality (which for utilitarians might be where most of the potential value is).
In a large enough multiverse, (1) seems likely. (Though if (1) is false and we're in a non-veridical simulation, it's unclear how much evidence we can have about whether we're in a large multiverse[2]).
Even if (1) did not hold, we might hope that some of observations are still useful in helping us influence the basement reality e.g. mathematical/logical/ philosophical arguments [3]. Of course simulators could be be arbitrarily Cartesian-demon like, but these seem safe to me to bracket-out.
I think (2) is not too tricky to accept and widelt accepted in these parts (though I think many academic decision theories would reject). Of the range of views the axes described in this post, one can be an evidentialist/EDT-like or use the algorithmic ontology.
Regarding veridicity, I think there's some risk of circularity here (and am not certain I've thread the needle):
With non-veridical simulations, I want to say something like ruling out simulations that some smart intelligence would say are of "unreasonable counterfactuals" (or are just complete fiction - i.e. they don't aim to capture some of the same dynamics as they believe they have in their own universe/multiverse)
So for example, if the totality of what exists ("the universe") was determined by some arbitrary (to some highly intelligent observer who understood physics as much as possible ) initial conditions, them I would class the simulation that runs "the universe but with different intial conditions" as reasonable.
With respect to an intelligent observer that believed in (and had good reason to believe) in a sufficiently large multiverse, then it's plausible that many counterfactuals are "reasonable".
Conditional on being in a non-veridical simulation, beyond our prior we have an anthropic update (if we are non-updateless) but little else to inform us about the world. (Perhaps our prior has expectations over which simulations that basement beings run)
Without (1), I don't think we can necessarily rule out simulators being able to run "counterlogical simulations" (this may only seem impossible to us because of the world they are happening to simulate).
Thanks Tristan, that is a cool framing! And yes I agree that if it's true, it nicely captures some of the lessons from my post in a psychologically easier way.
There are theoretical assumptions underlying this, like EDT over CDT as you point out. It also seems like you're assuming a large multiverse, which SIA likes a lot more than SSA. Do you endorse SIA as well?
A comment on how variations on the simulation hypothesis like this one may actually defeat the popular formulations of the simulation argument. Here's physicist Sean Carroll's formulation of the simulation argument (which, like other popular formulations, differs from philosopher Nick Bostrom's original argument):
We can easily imagine creating many simulated civilizations.
Things that are that easy to imagine are likely to happen, at least somewhere in the universe.
Therefore, there are probably many civilizations being simulated within the lifetime of our universe. Enough that there are many more simulated people than people like us.
Likewise, it is easy to imagine that our universe is just one of a large number of universes being simulated by a higher civilization.
Given a meta-universe with many observers (perhaps of some specified type), we should assume we are typical within the set of all such observers.
A typical observer is likely to be in one of the simulations (at some level), rather than a member of the top-level civilization.
Therefore, we probably live in a simulation.
Carroll observes that this formulation of the argument implies a contradiction, since typical simulations or universes will be bottom-level simulations that don't have the compute to create more simulations.
If we take some further series of reasoning steps from (7) to the view that we live in a simulation of a tiny fragment of the world that simulates a short span of years before it terminates, then we arrive at the conclusion that we will not be able to create simulated universes with many conscious beings in them. Therefore, (3) becomes false. (This is similar to Sean Carroll's more general point.)
Some series of steps — whatever they are— takes us from (7) to a particular conclusion. But then that conclusion loops back around and renders one of its premises false, thereby rendering the whole argument unsound. In this way, the argument is that we live in this particular kind of simulation — whatever that argument is — ends up defeating itself, assuming it rests on the sort of popular formulation of the simulation argument that I quoted from Sean Carroll.
For any version of the simulation argument to be coherent and valid (let alone sound), care needs to be taken with the logical steps that take us from the premises to the conclusion. Careless logic will create invalid arguments and wrong conclusions.
For instance, if we end up a positing top-level, non-simulated universe (or set of universes) fundamentally and radically different from our own, which operate according to different laws of physics and perhaps other different laws of nature or different metaphysical laws, then the simulation argument seems to fall apart. This prevents us from positing that the top-level universe (or set of universes) has infinite computation, unlike — apparently — our own.
If there are no constraints on the metaphysical or natural laws that rule the top-level universe, then we could just as easily say that the top-level universe is Heaven and God created our universe supernaturally. Or any of an infinite number of similar conceivable origin stories. (This is not an idea original to me, but I can't remember who I heard it from. Maybe the physicist David Deutsch? But I'm not sure.)
So, it seems important to the simulation argument's coherence and logical validity that a) our universe will actually create simulated universes eventually and b) that the top-level universe be similar to our universe, e.g., that it shares laws of physics, other laws of nature, and any other metaphysical laws that apply. It seems that any version of the simulation argument that doesn't meet both criteria (a) and (b) can't be sound.
On top of this, it should be said that it's completely arbitrary to suppose that a simulation would focus on North American and British effective altruists as opposed to, say, Japanese librarians or Sicilian clowns. If you're going to entertain the notion of a simulation that only focuses on a narrow slice of the world for a narrow time — which seemingly can't be reached by any logically sound argument — then there are endless possible versions of that simulation. There seems to be nothing that particularly supports this imagined version of a simulation above any of the endless other conceivable varieties.
I'm surprised that this is a featured post on the Effective Altruism Forum because it's such a bizarre, crazy idea and because there is no attempt to argue that it's true. It's just asserted. Moreover, the idea rests on the simulation argument, which itself is quite flawed. The idea that effective altruists are the only people who really exist (and that everyone else is just an illusion) is not an idea that could have any rational basis. It is really strange, then, to see the employees of the Centre for Effective Altruism who run the EA Forum single this post out as one that has particularly high quality. What on Earth is going on?
I wrote in my post "The sad decline of effective altruism" that effective altruism has suffered a steep decline in its intellectual standards over the last 15 years. It's possible that the featuring of this post was some kind of fluke or mistake, but it fits the pattern of what I've observed.
If someone earnestly believes that only effective altruists exist and 99.9%+ of people on the planet are fake, from a place or compassion and genuine concern, I would recommend they talk to a qualified mental health professional. This is outside the realm of a rational difference of opinion and into mental health territory.
I would be worried if anyone I loved told me they believed they're part of an elite group comprising the less than 0.1% of humanity that is real, while everyone else is fake and doesn't really exist. I would want them to get help. I extend that same concern to anyone who truly believes the idea entertained in this post.
I will probably not read or reply to further comments here, but I welcome civil, kind messages and comments on Substack.
References to the infohazard I describe below have now essentially been removed from the post. Thanks very much to Ariel for engaging in productive dialogue and considering my suggestions!
Original text:
I think one of the ideas that is heavily referenced in this post is a serious infohazard, even despite being presented carefully and alongside objections. I would recommend instead that all references to this idea be removed completely.
[Description of the infohazard now redacted.]
As an AI safety researcher, I still think this idea is very implausible, mostly because I think it implies one of several claims that are all very extraordinary from my perspective:
[Description of counter-arguments redacted.]
But even with all the hedging and counter-arguments, I can imagine this idea being intuitively compelling to some people, and leading to morally repugnant behaviour (including potentially in important decisions).
Edit: Why?
A lot of people here have a bias towards taking unusual ideas seriously / scout mindset.
As @Yarrow Bouchard 🔸 said elsewhere, strong objections aren't always forthcoming, even to shaky arguments. It took me a while to form a view on this argument myself.
Adding to Tristan's 'large enough multiverse' point, and re section iii ('We Might Get Terminated Soon'). There seem to be a few ways to think about personal continuity here. To describe two:
Continuity is causal, and termination of simulations contributes to some probability of "seeing nothing" next equal to the probability of being a simulated copy.
Continuity is content-based, and termination of simulations just means you continue to whatever remains and is sufficiently coherent to be your next moment, e.g. copies in base reality (related to Wei Dai's 'identical copy immortality').
I kinda lean toward 2 > 1. If there’s nothing special about causality that links you from one moment to the next, then what links them might just be that the later moment’s contents coherently follow on from the earlier one’s.
Continuity is ontologically tricky, but in practice we act as if there’s a fact about what we’ll experience next. Overall, I expect to see stuff even after simulations end.
That's a cool thought and I certainly hope you're right. I frankly don't know how to think about it. When I read about the teletransportation paradox, the fact that there's a copy of me doesn't relieve my anxiety much about my body being destroyed. But when I think about the fact that my brain's constituent atoms are replaced over time, I do feel a sense of continuity, but maybe that's an illusion? It's hard to know.
I have trouble understanding the different levels of fidelity argument. It seems like a belief that would have obvious evidence in the real world, eg some humans having more neurons, more complex neurons, different readings of brain activity, and less NPC behaviour. To such a degree that those types of brains/ people would be conscious while the other would not! All while having people in the other extreme of the threshold arguing that subatomic particles may be conscious.
I mean, if you care a lot about animals, knowing that some humans may be more complex than others could make you believe they are *more* conscious. Why would it move your threshold for binary consciousness?
I also haven't heard of scientifically minded people making those observations themselves (in neural stuff/NPC behaviour) and then trying to find an explanation for them. But it could be because some of these arguments imply that some people may be qualitatively better/worse than others and saying that is bad for multiple reasons.
And that could be the same reason why I don't see people arguing those differences between people just based on simulation theory, nor being interested in looking for evidence in favor of or against it (or at least sharing positive proof of it in public).
It doesn't make sense to ask questions like, Does a computer program of a mind really instantiate consciousness? That question reifies consciousness as a thing that may or may not be produced by a program. Rather, the particles constituting the computer just move—and that's it. The question of whether a given physical operation is "conscious" is not a factual dispute but a definitional one: Do we want to define consciousness as including those sorts of physical operations?
The relevant question is how much animals' experiences are discounted relative to the global poor's experiences. If they're discounted the same amount, animal welfare remains ~1000x as cost-effective [ratio between the cost-effectiveness of corporate campaigns for broiler welfare and GiveWell's top charities neglecting effects on non-target individuals].
I got this estimate based on the sentience-adjusted welfare ranges Rethink Priorities (RP) initially presented, and neglecting effects on soil invertebrates. However, I have little idea about whether chicken welfare reforms increase or decrease welfare accounting for uncertainty in welfare comparisons across species, and effects on soil invertebrates. For the sentience-adjusted welfare ranges presented in Bob Fischer's book, which are similar to the ones RP initially presented, I think effects on soil invertebrates would be much larger than those on chickens. So, trusting the values in the book, and excluding interventions decreasing uncertainty, I would focus on increasing the welfare of soil invertebrates by increasing or decreasing agricultural land. I believe this can be achieved more cost-effectively by the Good Food Institute (GFI) or Giving What We Can (GWWC) than interventions improving the conditions of farmed animals. In reality, I favour interventions decreasing uncertainty about welfare comparisons across species, and effects on soil invertebrates.
Hi, thanks Vasco! The Tomasik post is interesting, but he assumes a theory of consciousness which rules out lots of theories many people (including myself) find plausible like psychophysical laws, type B physicalism, and more.
On animals, my main view is that the vast majority of the effect of any near-term intervention is on animals rather than humans. I've come around to the idea that there are many more cost-effective animal interventions than THL, having donated to SWP and the Insect Institute. GFI is a strong choice as well. I agree soil invertebrates matter in expectation and that the animal welfare community should put some level of effort into incorporating them into cost-effectiveness estimates.
The Tomasik post is interesting, but he assumes a theory of consciousness which rules out lots of theories many people (including myself) find plausible like psychophysical laws, type B physicalism, and more.
Do you think lack of consciousness implies a moral weight of exactly 0? If so, how are you defining consciousness? For any sharp definition, infinitesimal changes in the state of an organism would make them go from not conscious to conscious. Hereare related discussions.
On animals, my main view is that the vast majority of the effect of any near-term intervention is on animals rather than humans. I've come around to the idea that there are many more cost-effective animal interventions than THL, having donated to SWP and the Insect Institute.
Do you endorse moral weights in the ballpark of the sentience-adjusted welfare ranges presented in Bob's book? This would be compatible with the above. However, for such moral weights, it is unclear to me whether electrically stunning farmed shrimps increases or decreases animal welfare. Do you disagree?
GFI is a strong choice as well.
Is this because of the effects on farmed animals or soil invertebrates?
The simulation argument is silly, for reasons I've previously detailed here and here. For one, the physicist Sean Carroll cleverly pointed out that the simulation argument contains a logical self-contradiction. Often, a logical self-contradiction is enough to end all discussion! Yet that's not even the end of the simulation argument's problems.
Another problem is that forecasts about future technological events within our universe don't necessarily tell us the story of the creation of our universe. Let's say we could accurately forecast that our descendants will simulate lots of universes. What does that tell us about the creation of our universe? Nothing, without further logical steps.
By analogy, let's say you plan to have a baby using IVF. Does that imply you were born thanks to IVF? No, of course not, not just by itself. Yet the simulation argument wants us to infer that if we create a universe a certain way, then our universe was also created that same way. This is an invalid inference.
But this still isn't the end of problems for the simulation argument, which just gets messier and more objectionable the more you scrutinize it. For instance, what major crimes or morally abhorrent acts might the creator of a simulation be guilty of? Torture, murder, genocide...? Wouldn't there be prohibitions against this sort of thing?
I could go on. There is just no good reason for the simulation argument to be taken seriously.
Edit (Sunday, October 18, 2026 at 8:50am UTC): I realized after writing this comment that I made a few minor mistakes, or at least there are a few minor things that should be clarified. These might be either boring and nipticky or important, depending on your perspective.
First, Nick Bostrom's original simulation argument is not an argument that we live in a simulation. It contains the possibility that simulations might be morally or legally prohibited. However, most people hear a version of the simulation argument different from what Bostrom originally argued. A good example is Sean Carroll's formulation. My point about moral or legal prohibition applies to the more popular formulations of the simulation argument, but not to Bostrom's original simulation argument, which already contains that point.
Second, my point about inferring the universe's origins from technological forecasts and whether it's a valid inference is already covered in Sean Carroll's blog post, which addresses the simulation argument's anthropic reasoning. The anthropic reasoning should be questioned, but even popular formulations of the simulation already contain the extra steps I was alluding to. In any case, it's worth thinking carefully what series of logical steps takes us from a technological forecast to a cosmological origin story. That was my point, but it was not clearly expressed.
It remains true that the popular formulation of the simulation argument discussed by Sean Carroll implies a logical contradiction. It remains true there are multiple major problems with that version of the argument. Nick Bostrom's original simulation argument is also objectionable, but less so because it is not an argument that we live in a simulation.
The bottom line remains that it's silly to think we live in a simulation. Neither Bostrom's original argument or the more popular reformulations present a serious case that we do.
I haven't read any replies to this comment and probably won't because I usually find discussions on the EA Forum to be unconstructive. However, I welcome further discussion on Substack.
Thanks for engaging. I'd push back strongly on these points. In any case, my post is just saying the AGI sim hypothesis is plausible enough to merit discussion of its implications, so the bar for my post is just that it's not silly enough to dismiss.
You frame Carroll's argument as "enough to end all discussion", but not even Carroll seems to believe that--he demonstrates thoughtful uncertainty and says "likely there are standard responses", and indeed there are. The standard response is that deeper sims don't need to be incapable of running sims, they can just get shut off before they do. Bostrom anticipated this argument in his original paper, writing that nested sims are a reason why we'd expect ours to be "terminated when we are about to become posthuman". This scenario is one of the main implications of the AGI sim hypothesis I discuss in my post above. There are other responses to Carroll's argument too.
On the IVF analogy, the simulation argument says that if we get evidence that most observers with experiences like ours are simulated, unless we have evidence to the contrary, we should think we are also simulated. The idea that our descendants or AGI would run sims of AGI origins, which would create lots of observers with experiences like ours, is evidence that most observers with experiences like ours are simulated. In contrast, planning to use IVF is not evidence that you yourself are an IVF baby.
On the ethical and opportunity cost arguments, your arguments are evidence for one of my post's main scenarios -- the simulation being run by a misaligned AGI -- rather than evidence against the simulation hypothesis altogether. They have some force, which is why I personally think the misaligned AGI simulator is the more likely one, but I don't think they push that hard. Humans tolerate an immense amount of preventable suffering today (global poverty, factory farming, wild animal suffering). I wouldn't be that surprised if a "human-aligned" AGI would run simulations like ours (if we are indeed a sim).
On theism, I agree there are some resemblances, specifically the claim that the universe was made for a reason. But the evidence for the hypotheses differs tremendously. The simulation hypothesis is built from an expectation of future sim-running capabilities we can extrapolate from known physics. If we learned that there was a way to create pocket universes whose inhabitants worshipped us, that rightly should increase our credence in theism. That's the situation I think we're in wrt the simulation hypothesis. There's also a lot of specific things the AGI sim hypothesis predicts that theism doesn't: us being in the AI transition, an impersonal simulator rather than a personal god, etc.
(Edited to redact some material commenters considered infohazardous.)
If you spend too much time talking to EAs or rationalists, someone will raise the possibility that we’re living in a simulation. I’ve met people who, like Elon Musk, are pretty convinced we are, and I’ve met those who, like Joe Carlsmith, are more skeptical. Historically, the usual argument was that we’re being simulated for entertainment. As AI capabilities have increased, the argument I’ve more often heard is that the simulators are interested in AGI origins (an “AGI sim”).
You’re an EA/rationalist who’s at most one step removed from people who’ll play a pivotal role in AGI origins. If the simulators are compute constrained, you being you is best explained by a simulation rendering reality at a fidelity proportional to what the simulators care about. In other words, the vast majority of people and almost all animals might not be conscious at all.
If the simulators are compute constrained, they’ll probably just terminate the simulation once they’ve gathered the information they want. Whether or not our x-risk reduction efforts go well, we might be doomed.
I think there are many reasonable counterpoints to the above, but I personally find this account of our reality plausible, and I’ve met other EAs who feel the same way. If we took these conversational curiosities seriously, what would that imply for EA?
A. The Simulation’s Crucial Considerations
i. Why An AGI Sim?
In brief, we should think we might be in a simulation if we think advanced intelligence (perhaps our descendants) with sufficient computational resources would be likely to run many simulations of history.
Historically, the typical explanation for why they’d simulate us was entertainment. I wasn’t very convinced by this, because it just didn’t seem like our reality was optimized for entertainment. If we’re entertainment, why is history only pretty entertaining but not maximally? Do our lives seem particularly entertaining? Why is fundamental physics not simpler and more optimized for entertainment? Why the heck did they create a huge particle that does nothing but disappear in 0.3 picoseconds? Why does one of the fundamental forces care whether you’re going left or right? It just doesn’t make sense to me.
As an AGI expands its reach over its lightcone, it might want to understand what kinds of AGIs it might bump into as it expands, so it can be prepared and potentially bargain with them.[1]
A human-aligned AGI might want to run counterfactual Earths to better understand what it should value.
This would also explain why fundamental physics is as weird as it is (maybe it’s similar to that of the simulator's reality), why our world doesn’t look obviously simulated, and why you’re an EA/rationalist who’s at most one step removed from people who’ll play a pivotal role in AGI origins. There’s lots one could quibble with here, but for the purposes of this post, let’s assume we’re in an AGI sim, so we can examine which conclusions would follow.
ii. Some Might be Simulated at Lower Fidelity
Especially if the simulator's reality is like ours, they might be compute constrained. Simulating a whole universe is way more expensive than rendering the sim at a fidelity proportional to what you care about. For many of us, proportional fidelity would also explain why we are who we are.
This hypothesis would predict people further from AGI to be rendered at lower fidelity, so their conscious experiences would be less detailed. Animals, irrelevant to the AGI transition, might have even less detailed experiences, or possibly not be conscious at all. (Maybe for the best.)
There’s a lot of pushback one could give here:
I observe myself as conscious, but instead of being Jensen Huang, I’m a quant from Boston who writes essays about chickens—not exactly the optimal allocation of simulation compute. In general, the further you are from the AGI transition, the less convincing this part of the post will be to you. If you’re in Nigeria handing out malaria nets, you’re rolling your eyes.
There might be no way to cheaply approximate billions of people when those people are the AI’s training data, the demand AIs are built to satisfy, and an influence on future AI values.
More speculatively, there’s no way to know whether the simulator's reality’s physics has a notion of compute as a constraint the way our reality does. If they don’t, there’s no reason why the sim couldn't just be run at full fidelity.
Finally, history teaches us that the reasoning procedure behind the conclusion that “my in-group matters but outsiders don’t” has a dismal base rate. Atrocities like slavery, factory farming, etc remind us that the costs of getting this wrong vastly outweigh the gains.
iii. We Might Get Terminated Soon
If the simulators are computationally constrained, we should expect them to terminate the sim once they’ve learned what they wanted to know.
As always, there are counterarguments. If AGI values take time to settle, maybe the simulator will run our universe for a few million more years until the AGI converges on a stable strategy. If the simulator's reality isn’t compute constrained, maybe this universe will just keep running.
The main implications here are for longtermism. Longtermists have argued (correctly, in my view) that applying an ethical discount to the far future is philosophically dubious. An uncertain termination date implies an empirical discount to the far future. This discount's implications would be similar to those of an ethical discount.[2]
B. Implications for EA
i. Fanaticism Doesn’t Care
If you’re fully fanatical about expected value and endorse maximizing expected choiceworthiness, most of the above shouldn’t change what you do. Even if there’s an astronomically small chance we’re not in a simulation, the prospect of creating 10^50 happy lives in the far future still dominates. The exception, which I'll discuss in more detail below, is trying to influence the simulator's reality, which could be worth even more.
ii. The EA Decision Tree
If you lean towards a moral parliament, are risk averse, or just have ordinary human scope sensitivity, the situation becomes more complicated.
The two crucial considerations from section A give us a decision tree. Do the simulators render everyone at full fidelity, or at a fidelity proportional to what they care about? And do they terminate us soon, or let the sim keep running?
Full fidelity, sim terminated soon: If everyone is conscious but we'll be terminated soon, neartermism (global health and animal welfare) would likely be a better use of resources than x-risk or other longtermist work. Within neartermism, I've argued that animal welfare would be the most cost-effective use of resources.
Full fidelity, sim keeps running: Standard EA; keep doing what you've been doing.
Proportional fidelity, sim terminated soon: The global poor and animals get discounted, but the scale of animal suffering is large enough that animal welfare plausibly still looks better unless animals are hardly rendered at all.
Proportional fidelity, sim keeps running: Everything looks worse here, but neartermism is discounted more than longtermism. On the margin, this should move you toward longtermism.
iii. Proportional Fidelity's Effect on Neartermism
If animals are rendered at much lower fidelity, the scale of animal suffering could be much lower than it appears. While this would reduce the importance of animal welfare, animal welfare would plausibly still look better than global health.
The relevant question is how much animals' experiences are discounted relative to the global poor's experiences. If they're discounted the same amount, animal welfare remains ~1000x as cost-effective. However, animals probably would be discounted more than the global poor. The global poor affect the AI transition by providing AI training data and economic demand, but it's hard to come up with a convincing account of how an animal's conscious choices would materially affect the AI transition. So the scale gets less overwhelmingly in favor of animals, but unless animals are discounted up to 3 orders of magnitude more than the global poor, animal welfare still looks like a better use of resources.
Whether or not you take proportional fidelity seriously, global health and animal welfare still matter a lot in expectation. Over 800 million people live in extreme poverty, tens of billions of animals are being factory farmed at any given time, and far more suffer in the wild. There isn't a reasonable discount which stops these from being (in expectation) among the world's most important problems.
iv. Compute Fungibility
If we weren't being speculative enough, let's dial it up to 11. Assuming the simulators are compute constrained, anything that makes our sim cheaper frees up compute for other sims. If you prevent lots of wild animals from existing, that reduces the compute our world uses, and that compute gets reallocated to other worlds which have their own suffering wild animals. To generalize, if you have a good reason to think our sim is better / worse than other sims, on the margin, this would nudge us towards increasing happiness / reducing suffering.
v. Influencing The Simulator
If the simulator is human-aligned and running counterfactual Earths to understand what it should care about, then being legible about what we value could help the simulator do much greater amounts of good in their reality than is possible in our own. Alternatively, if the simulator is misaligned but is simulating us because it wants to understand what other values are out there, being legible about our values makes our simulator more likely to cooperate and do mutually beneficial trade with beings in its reality who are like us.[3] Work on shaping AI values in our reality would be one direct way to target these benefits. This would hold even if our sim will be terminated soon or is being run at proportional fidelity.
Unfortunately, there's an s-risk concern which counterbalances this. If it's easier to create suffering than happiness, or especially if you're a negative utilitarian, it's extra bad to tell a misaligned AGI what it can do to get leverage over beings like us.
vi. We Are Digital Minds
If we're in a sim and you think we matter, that's an existence proof that digital minds (ourselves) matter morally. On the margin, if you aren't already sold on the importance of digital mind welfare, the simulation hypothesis is one reason to be supportive.
C. Testing the AGI Simulation Hypothesis
There are a few ways we could get weak evidence for whether or not the AGI simulation hypothesis is true:
The AGI sim hypothesis predicts short timelines. If you're reading this post fifty years later and AGI turned out much harder to achieve than we thought and it doesn't seem likely anytime soon, that's evidence against this hypothesis.
The AGI sim hypothesis predicts there's high uncertainty about how the AGI transition will go. If our simulator is running Monte Carlo simulations, they'd concentrate the runs where there's the most variance, since it's less worthwhile to run sims where the outcome is more obvious. It is literally true that Trump being President is evidence for the simulation hypothesis. The more undetermined you think the future is, the more this should shift you towards the simulation hypothesis.
D. Your Childhood Might Not Have Happened To You
If the simulator is compute constrained, it's wasteful to run your childhood, since it isn't as relevant to the AGI transition. Instead, they might just run one sim of the world when you were growing up, save that sim's state at some checkpoint, and spin up a larger number of sims from that checkpoint state. This is similar to how you can fork an LLM conversation at the tenth turn and the forked LLM has the memory of the earlier turns even if it didn't personally experience them.
This model of the world says it's very unlikely that the instance of you who's reading this actually experienced your childhood. Even fewer instances, or maybe none at all, would have experienced humanity's earlier history.
E. So What Should We Actually Do?
If we're in an AGI sim, your childhood might not have happened to you, but the present moment where you're reading this is a real moment the simulator is watching. The value of public conversations and the importance of digital minds are two of the AGI sim hypothesis' most robust implications.
If you're an altruist and you put any credence in the AGI sim hypothesis, I encourage you to think out loud about what the credences you put in the above EA Decision Tree imply about your priorities. In a world that’s being watched, reasoning in public might be one of the most valuable things you can do.
Note: This post was crossposted from the Coefficient Giving Farm Animal Welfare Research Newsletter by the Forum team, with the author's permission. The author may not see or respond to comments on this post.
Subtitle: On vegan advocacy, effective altruism, and FIFA
I turn 40 today. Here are some hot takes.
On factory farming
1. Our biggest challenge is salience. If factory farming led the evening news, it wouldn’t last lo...
When speaking to candidates who are considering working with us at 80,000 Hours, I often hear people worry about replaceability.
The familiar version of the worry is about whether the organisation will hire someone else in your place. If it will, then the difference you make by taking the job is only the difference between you and them.
There are a couple of well-known answers to that. The person hired in your place probably cares about the world too, and would go on to do somethi...
> TL;DR: No, but you might, if you leave it too late: We’re hiring for seven roles, and the people who join now will shape what a much bigger AIM becomes. The first step is cheap -- an expression of interest form.
My worry
I rejoined...
Some key actionable ideas that weren't discussed in the post: