I am happy to announce a new paper I co-wrote with Anders Sandberg, which is now a public preprint (PDF). This is a link-post for that paper, and to a post on Lesswrong that contains a summary of some of the arguments.
Abstract: How much value can our decisions create? We argue that unless our current understanding of physics is wrong in fairly fundamental ways, there exists an upper limit of value relevant to our decisions. First, due to the speed of light and the definition and conception of economic growth, the limit to economic growth is a restrictive one. Additionally, a related far larger but still finite limit exists for value in a much broader sense due to the physics of information and the ability of physical beings to place value on outcomes. We discuss how this argument can handle lexicographic preferences, probabilities, and the implications for infinite ethics and ethical uncertainty.
34
34
Reactions
Thanks David, this looks like a handy paper!
Given all of this, we'd love feedback and discussion, either as comments here, or as emails, etc.
I don't agree with the argument that infinite impacts of our choices are of Pascalian improbability, in fact I think we probably face them as a consequence of one-boxing decision theory, and some of the more plausible routes to local infinite impact are missing from the paper:
- The decision theory section misses the simplest argument for infinite value: in an infinite inflationary universe with infinite copies of me, then my choices are multiplied infinitely. If I would one-box on Newcomb's Problem, then I would take the difference between eating the sandwich and not to be scaled out infinitely. I think this argument is in fact correct and follows from our current cosmological models combine with one-boxing decision theories.
- Under 'rejecting physics' I didn't see any mention of baby universes, e.g. Lee Smolin's cosmological natural selection. If that picture were right, or anything else in which we can affect the occurrence of new universes/inflationary bubbles forming, then that would permit infinite impacts.
- The simulation hypothesis is a plausible way for our physics models to be quite wrong about the world in which the simulation is conducted, and further there would be reason to think simulations would be disproportionately conducted under physical laws that are especially conducive to abundant computation
The simulation hypothesis is a plausible way for our physics models to be quite wrong about the world in which the simulation is conducted, and further there would be reason to think simulations would be disproportionately conducted under physical laws that are especially conducive to abundant computation
The main reason for taking the simulation hypothesis seriously is the simulation argument, but that argument needs to assume that our physical models are broadly correct about reality itself and not just the "physics" of the simulation. Otherwise, there would be no warrant for drawing inferences from simulated sense data about the behavior of agents in reality, including whether these agents will choose to run ancestor simulations.
There is some effect in this direction, but not a sudden cliff. There is plenty of room to generalize, not an in. We create models of alternative coherent lawlike realities, e.g. the Game of Life or and physicists interested in modeling different physical laws.
Thanks, and thanks for posting this both places. I've responded on the lesswrong post, and I'm going to try to keep only one thread going, given my finite capacity to track things :)
Thank you so much for this paper! I literally made a similar argument to someone last weekend (in the context of economic growth), glad to have a canonical/detailed source to look at so I can present more informed views/have an easy thing to link to.
I will read the rest of the paper later, but just flagging that I don't find your response to "incomplete understanding of physics" particularly persuasive:
Perhaps our understanding of physics is incorrect. That is, it is possible that our understanding of any of the assumed correct disciplines discussed here, from cosmology to computation. This is not merely an objection to the authors’ personal grasp of the subjects, but a claim that specific premises may, in the future, be found to be incorrect.
I think the strongest version of the "we don't understand physics" argument is that we (or at least I) have nonzero credence in physics as we know it to be mistaken in a way that allows for infinities. This results in an infinite expected value.
Now, perhaps we can exclude arbitrarily exclude sufficiently small probabilities ("Pascal's mugging"). But at least for me, my inside-view credence in misunderstanding the finitude of physics is >0.1%, and I don't think Pascal's mugging exceptions should be applicable to probabilities at anywhere near that level.
Michael Dickens has a different issue where finite distributions can still have infinite expected value, but I have not read enough of your paper to know if it addresses this objection.
Thanks. I agree that we should have non-infinitesimal credence that physics is wrong, but to change the conclusion, we would need to "insist that modern physics is incorrect in very specific ways." Given the strength of evidence about the existence of many of the limits, regardless of their actual form or value, that is a higher bar. I also advise looking closely at the discussion of the "Pessimistic Meta-induction," and why we think that it's reasonable to be at least incredibly confident that these limits exist.
That doesn't guarantee their existence. But after accepting a non-zero credence in those specific types of incorrect theory, we need to pin our hopes for infinite value on those specific occurrences; we would need to maximize expected value conditional on that very small probability in order to find infinite value, and neglect the very large but finite value we are nearly certain exists in the physical universe. That seems difficult to me.
Thank you so much for this paper! I literally made a similar argument to someone last weekend (in the context of economic growth), glad to have a canonical/detailed source to look at so I can present more informed views/have an easy thing to link to.
I will read the rest of the paper later, but just flagging that I don't find your response to "incomplete understanding of physics" particularly persuasive:
I think the strongest version of the "we don't understand physics" argument is that we (or at least I) have nonzero credence in physics as we know it to be mistaken in a way that allows for infinities. This results in an infinite expected value.
Now, perhaps we can exclude arbitrarily exclude sufficiently small probabilities ("Pascal's mugging"). But at least for me, my inside-view credence in misunderstanding the finitude of physics is >0.1%, and I don't think Pascal's mugging exceptions should be applicable to probabilities at anywhere near that level.
Michael Dickens has a different issue where finite distributions can still have infinite expected value, but I have not read enough of your paper to know if it addresses this objection.
Thanks. I agree that we should have non-infinitesimal credence that physics is wrong, but to change the conclusion, we would need to "insist that modern physics is incorrect in very specific ways." Given the strength of evidence about the existence of many of the limits, regardless of their actual form or value, that is a higher bar. I also advise looking closely at the discussion of the "Pessimistic Meta-induction," and why we think that it's reasonable to be at least incredibly confident that these limits exist.
That doesn't guarantee their existence. But... (read more)