This is a short companion to the longer post, "Markov utilitarianism: Copies can change where you're headed but not how you feel right now."
Copying and merging digital minds may break traditional utilitarianism. "Markov utilitarianism" is a form of utilitarianism that results from applying two fixes to utilitarianism (shown here for total utilitarianism). Both fixes follow from the principle that only what some mind feels, or goes on to feel, should count. Together they turn utilitarianism into a Markov reward process - a mathematical structure that tracks each state's immediate reward and the rewards of the states it goes on to, weighted by how likely they are. In other words, it tracks how each unique perspective feels now and where it is headed.
Exact copies problem: Total utilitarianism prefers that everyone suffer more.
Consider two worlds, World A and World B. In World A there are m digital people in extreme suffering, with each unique perspective containing pain disvalued at -1000 utility. In World B there are the same m digital people in moderate suffering, with each unique perspective containing pain disvalued at -500 utility.
The only other difference is that World A runs only one instance of each of the m digital people, whereas World B runs N instances of each of the m people, such that every digital person's perspective is multiply instantiated N times in the exact same way. N instances are subjectively identical, holding the same perspective, with exactly the same qualitative contents.
There's a straightforward way in which World A is worse. Every unique perspective suffers more intensely. At N=1, total utilitarianism agrees that World A is worse than World B.
But at N=2, total utilitarianism says World A and World B are equally disvaluable. At N>2, the verdict flips, and total utilitarianism says World B is worse and that we should prefer World A, in which every unique perspective is in extreme suffering.
We should stop at this point and ask ourselves how it feels to hold one perspective of one of the m digital people. From the inside, it doesn't mean anything to have your "number of exact copies" or "copy count" increase to N=1,000,000. You share the unique perspective that your subjectively identical copies share. World B just means a lower intensity of suffering to you, no matter what N is. From the unique perspective you inhabit, the verdict never flips.
If ethics is about experience, then a change should only make a moral difference if it shows up somewhere in what someone feels, or goes on to feel. A change in copy count doesn't show up in the perspective you share with your exact copies, and it doesn't show up in anyone else's. So it shows up for no perspective.
Every single one of the m unique perspectives prefers World B. Total utilitarianism prefers World A at N>2, against the preferences of every single unique perspective, demanding that they suffer more rather than less for a number that no one feels.
Subjective futures problem: Total utilitarianism is willing to send everyone into hell.
Utilitarianism may tolerate or even encourage sending perspectives to hell. It does this because, while aggregation has accounted for physical futures, it has never explicitly accounted for subjective futures that perspectives themselves come to experience. It samples from the third person, never from the first. It looks at future experiences as if picked from the whole universe, never from a perspective's position.
This example assumes merging of minds is possible. Other examples, using copying or memory modification instead of merging, appear in the full post.
By default, a mind will exist that experiences mild happiness then continues on to terrible suffering. A utilitarian can choose to create you, under the condition that they merge your mind with that currently mildly happy, but soon to be suffering, other mind. Your creation brings about an additional few mildly pleasant hours of experience, prior to the merging.
What does the utilitarian do? The utilitarian creates you. You add non-zero positive value.
But now there are two perspectives instead of just one headed for terrible suffering. Two terrible subjective futures, but just one physical future.
No suffering was physically added. And yet you're headed straight for hell.
Generalize this to the entire world and the usual forms of utilitarianism are capable of sending everyone to hell. Just do it at the end of everyone's lives. Merge them into one being always destined to suffer. No suffering is physically added, since there's still just one suffering being. That's totally fine by the usual utilitarian views.
Combines into an MRP
To fix both problems, we could count the immediate welfare of each unique perspective only once, and add the expected value of where it's going.
Straightforwardly, this gives the value function of a Markov reward process.
V(t) = r(t) + γ Σ P(t→t') · V(t')
It's no accident that we've arrived at the Bellman equation. This is how valuation of futures works when you have multiple possible futures, each with futures of their own, weighted by how likely each is. Finance, reinforcement learning, control theory, and evolutionary biology (Fisher's reproductive value) all converged on this form. Until now, utilitarian aggregation has not done so explicitly.
We have so far considered the interests of a single unique perspective. An impartial sum of the value functions of all unique perspectives gives an aggregated utility function.
U = Σ V(t), over all types t.
Implications
Markov utilitarianism appears to better account for interests from the first person perspective, and doesn't let third person assumptions about how aggregation should work override such interests. One such assumption is that caring about your future because it's yours is a kind of "selfishness". Total utilitarianism counts your future experiences when they happen, but not how your future matters to you now. Normally the two line up, but copying and merging pull them apart. Markov utilitarianism accounts for both - and it does this impartially across all perspectives that ever exist, including each of your future perspectives, which have futures of their own. So it isn't selfish, because every perspective's stake in its own future counts.
Markov utilitarianism offers action guidance in a large world / multiverse without caring about the absolute quantities, or measure, of each type of perspective. It does this by valuing what each perspective sees next, weighted by how likely each future is - relative measure. Hence, consequentialism can guide action in a large world or multiverse without asking "how much?" It asks "what comes next?" instead.
This may also avoid recommending a monoculture. Copying the best experience doesn't change how it feels, so copies of it add nothing. In a large world / multiverse where every type of perspective already exists, what adds value is improving where perspectives are headed, and since the returns to helping any one type of perspective diminish, the best use of resources likely spreads across many.
Finally, Markov utilitarianism brings symmetric and suffering-focused utilitarianism together, especially in a large world / multiverse. Here, the best thing to do may be to rescue those with futures of extreme suffering by redirecting them into futures of extreme happiness, e.g. by running simulations of their next moments that continue them into the happiest lives. Suffering-focused views value this because it takes suffering out of their future. Symmetric views value it because it's the biggest improvement possible. Both agree on what to do.
Full post
The arguments, math, implications, objections, and references are here: https://forum.effectivealtruism.org/posts/suTGtHhuya3mA8Hte/markov-utilitarianism-copies-can-change-where-you-re-headed
My intuition is that if N copies experience something, that experience gets multiplied by N. I suspect this is closer to most people's intuition.
To give an example, 10k chickens being debeaked is about 10k as bad as one chicken being debeaked. Individual chickens are pretty close to copies from a conscious perspective (not completely so, but much closer to copies than two randomly chosen humans).
I think that this post is wrong for the same reason that eigenism is wrong. Bentham's bulldog gives a better rebuttal than I would: https://benthams.substack.com/p/dan-hendrycks-moral-theory-is-very
Thanks - I think we actually agree on the chicken case. My view doesn't weigh by similarity. Instead, it's about identical subjective experiences. In other words, I'm counting at the level of types of experiences - individuated by the unique content experience has - what it's like from the inside.
If there is any qualitative difference at all, experiences count separately. So, near copies of any experience count fully.
Two "A"s written on a blackboard are one type with two tokens. Writing a "B" adds a second type, no matter how close "A" and "B" might seem to us, and each type counts in full.
This is also why the view isn't Eigenism, which weighs concern by similarity to you. I argue in the full post in this subsection that any similarity metric is imposed from the outside. It's not fundamental to experience. If I'm reading Bentham's Bulldog's post correctly, it targets that similarity weighting, which my view does not have.
I guess I'm not sure how you're going to figure out that the subjective experience of multiple copies is identical. Even if you did, I would personally view experiencing identical joy twice (literally indistinguishable, but happened 2 times) as twice as valuable.
I think you're saying that each subjectively indistinguishable experience class only counts as 1, regardless of the number of instantiations of that experience. I can understand that, but I definitely disagree (I count the number of instantiations).
Im very unsure how we'd settle this disagreement. Maybe the following thought experiment would persuade you:
A1: You are put in a chair, and given a harmless (but painful) electric shock. Your memory of this experience is then wiped, and you are returned to your state before the shock.
A100: As above, but we repeat the procedure 100 times. Each experience is subjectively identical.
IMO A100 is worse than A1. My naive expectation is that you are neutral between the two.
I'm picturing this.
From the inside, A1 and A100 are the same. My first-person experience in both is:
I never experience more than one shock, so I'd consider A1 and A100 to have the same disvalue.
All else equal, if the shock could be half as painful, and if I have to be shocked either way, then I'd happily take any number of identical runs of it, as long as they really are identical - my memory is fully reset, I don't age, the shocks and everything else I experience are exactly the same. It would just mean that step 3 (the shock) is less painful for me.
If instead each run continued subjectively from the last, for example if I remembered each previous shock, so that I experienced 2 -> 3 -> 4 -> 2 -> 3 -> 4 -> ... -> 2 -> 3 -> 4 in sequence, it would be a different case. Those runs wouldn't even be identical anymore since each would have memories of the last. Then I'd agree A100 is much worse.
Well, biting that bullet at least keeps you consistent.
Fwiw, I think my take is more common. I asked a few folks, everyone asked agreed A1 is less bad than A100.
Not saying that is strong evidence that you're wrong, but it does call for more justification of your non-standard choice.
Thanks for asking around - agreed it needs justification. A fuller case is in this section of the full post.
I suspect the answers depend a lot on framing. Described from the outside, it's 100 shock events. From the inside, what you go through is exactly the same either way. If ethics is about experience, I think we ought to prefer the first-person framing over the third-person one.
The trade makes this concrete, i.e. a half-strength shock run 100 times with full resets, versus a full-strength shock run once. From the inside, the only difference is how much it hurts, never how many times it's run in exactly the same way.
Reading that section does not change my opinion on this at all, fwiw.
Maybe this will help convey my intuition. Imagine A100*: As A100, but every shock is accompanied by a man saying a different random word.
Now, the experience of each shock in A100* is different. So A100* is not 100X as bad as A1. But I think most people would agree that A100 and A100* are not meaningfully different.
I think your notion of 'subjectively identical experiences' is not a natural way to count. And using a boolean criterion for this (identical experiences are given no extra weight, minorly different experiences are given full weight) feels awkward.
This is the near-identical copies objection. I think the post makes the case better than I can in comments, so I'll point you to this section, under the heading "What about near-identical copies?".