One elephant in the room throughout my geometric rationality sequence, is that it is sometimes advocating for randomizing between actions, and so geometrically rational agents cannot possibly satisfy the Von Neumann–Morgenstern axioms. That is correct: I am rejecting the VNM axioms. In this post, I will say more about why I am making such a bold move.
A Model of Geometric Rationality
I have been rather vague on what I mean by geometric rationality. I still want to be vague in general, but for the purposes of this post, I will give a concrete definition, and I will use the type signature of the VNM utility theorem. (I do not think this definition is good enough, and want it to restrict its scope to this post.)
A preference ordering on lotteries over outcomes is called geometrically rational if there exists some probability distribution P over interval valued utility functions on outcomes such that L⪯M if and only if GU∼PEO∼LU(O)≤GU∼PEO∼MU(O).
For comparison, an agent is VNM rational there exists a single utility function U, such that L⪯M if and only if EO∼LU(O)≤EO∼MU(O).
Geometric Rationality is weaker than VNM rationality, since under reasonable assumptions, we can assume the utility function of a VNM rational agent is interval valued, and then we can always take the probability distribution that assigns probability 1 to this utility function.
Geometric Rationality is strictly weaker, because it sometimes strictly prefers lotteries over any of the deterministic outcomes, and VNM rational agents never do this.
The VNM utility theorem says that any preference ordering on lotteries that satisfies some simple axioms must be VNM rational (i.e. have a utility function as above). Since I am advocating for a weaker notion of rationality, I must reject some of these axioms.
Against Independence
The VNM axiom that I am rejecting is the independence axiom. It states that given lotteries A, B, and C, and probability p, A⪯B if and only if pC+(1−p)A⪯pC+(1−p)B. Thus, mixing in a probability p of C will not change my preference between A and B.
Let us go through an example.
Alice and Bob are a married couple. They are trying to decide where to move, buy a house, and live for the rest of their lives. Alice prefers Atlanta, Bob prefers Boston. The agent I am modeling here is the married couple consisting of Alice and Bob.
Bob's preference for Boston is sufficiently stronger than Alice's preference for Atlanta, that given only these options, they would move to Boston (A≺B).
Bob is presented with a unique job opportunity, where he (and Alice) can move to California, and try to save the world. However, he does not actually have a job offer yet. They estimate an 80 percent chance that he will get a job offer next week. Otherwise, they will move to Atlanta or Boston.
California is a substantial improvement for Bob's preferences over either of the other options. For Alice, it is comparable to Boston. Alice and Bob are currently deciding on a policy of what to do conditional on getting and not getting the offer. It is clear that if they get the offer, they will move to California. However, they figure that since Bob's preferences are in expectation being greatly satisfied in the 80 percent of worlds where they are in California, they should move to Atlanta if they do not get the offer (pC+(1−p)B≺pC+(1−p)A).
Alice and Bob are collectively violating the independence axiom, and are not VNM rational. Are they making a mistake? Should we not model them as irrational due to their weird obsession with fairness?
Dutch Books and Updatelessness
You might claim that abandoning the independence axiom opens up Alice and Bob up to get Dutch booked. The argument would go as follows. First, you offer Alice and Bob a choice between two policies:
Policy CA: California if possible, otherwise Atlanta, and
Policy CB: California if possible, otherwise Boston.
They choose policy CA. Then, you reveal that they did not get the job offer, and will have to move to Atlanta. You offer them to pay you a penny to instead be able to move to Boston. In this way, you extract free money from them!
The problem is they don't want to switch to Boston, they are happy moving to Atlanta. Bob's preferences are being extra satisfied in the other possible worlds where he is in California. He can take a hit in this world.
If California did not exist, they would want to move to Boston, and would pay a penny to move to Boston rather than Atlanta. The problem is that they are being updateless. When they observe they cannot choose California, they do not fully update on this fact and pretend that the good California worlds do not exist. Instead they follow through with the policy that they agreed to initially.
We can take this further, and pretend that they didn't even consider Atlanta vs Boston. They just got a job offer, and decided to move to California. Then all the world saving money disappeared over night, the job offer was retracted, and Alice and Bob are newly considering Atlanta vs Boston. They might reason, that if they would have taken the time to consider this possibility up front, they would have chosen Atlanta, so they follow through the policy that they would have chosen if they would have thought about it more in advance.
They have a preference for fairness, and this preference is non-local. It cares about what happens in other worlds.
I gave the above example about a married couple, because it made it cleaner to understand the desire for fairness. However, I think that it makes sense for individual humans to act this way with respect to their various different types of preferences.
