Why complete cluelessness is counterintuitive to me

I mean, specifically, this kind of pervasive cluelessness, where you can't justifiably decide between any two actions. It seems that such pervasive cluelessness is in tension with the idea of instrumental convergence. I'm personally sympathetic to imprecise probabilities, but intuitively I'm not convinced that cluelessness is that bad that we can't justify even basic learning and other (supposedly convergent) instrumental strategies, such as (at least) those in the low-footprint capacity building category.

And I think, if we have to show that cluelessness is at least not that pervasive, it seems useful to look at the most seemingly absurd cases. Being clueless about whether epistemic improvement is worthwhile is one of them. If we (or other aligned agents that we create) can in principle be non-clueless about some specific class of strategies, for instance, why wouldn't we at least sometimes be justified in taking actions that would make us non-clueless about them?^[1] (hence, being additionally non-clueless about whether this transition is better than doing nothing or pursuing some other alternative with a small opportunity cost) And if we are indeed justified in doing so, why can't we derive some broader system of instrumental strategies from this fact?^[2] (hence, potentially, having even more non-clueless states) It sounds strange that even strategies aimed at becoming non-clueless wouldn't be better justified than doing something random.

I have to admit, the situation is much less obvious than I first thought. Interestingly, some imprecise probabilists may have to pay to avoid free knowledge (unlike precise ones). And there is a thing called dilation: new information may make your intervals wider than before. But most importantly, knowledge is not actually free. And it's yet unclear to me how we should account for long-term changes in the structure of our entire decision trees in general. This seems to involve the topic of sequential rationality.

Anyway, why is impartial altruism so different from other value systems in this respect?^[3] DiGiovanni writes:

...suppose that instead of spending some of your free time reflecting on virtue ethics, you reflect on whether some intervention could be robust to unawareness. Maybe this shapes the salience of different strategies to your future self, and you end up attempting an intervention that’s worse than what you would’ve done by default.

In principle, we can come up with examples where learning goes wrong, but it's not yet clear why we should treat them as severely undermining the whole idea of epistemic improvement.

Intuitively, these side effects may seem like hand-wringing nitpicks. But I think this intuition comes from mistakenly privileging intended consequences, and thinking of our future selves as perfectly coherent extensions of our current selves.

But we don't have to think that our future selves are such perfectly coherent extensions. They are indeed different, but they may differ to a degree small enough for us to make a justified decision.

So far, it is not clear why impartiality is special. Learning seems to have broadly similar structures across many domains. Impartial altruism doesn't seem to be an exceptional domain such that some magical demons suddenly appear from nowhere to make you go mad if you know too much.^[4] Then why aren't there some learning strategies available to us that are more justified than doing nothing?^[5]

We’re trying to weigh up speculative upsides and downsides to a degree of precision beyond our reach. In the face of this much epistemic fog, unintended effects could quite easily toggle the large-scale levers on the future in either direction.

This seems to be relevant when deciding between some direct interventions (such as donations), but how does the associated cluelessness infect learning itself—which is supposed to help us compare them? Cluelessness may emerge here if we don't know how to compare learning with its alternative, which is possibly higher-EV. But what if the alternative we consider is something which has low direct impact on your environment, such as resting? Then by choosing to learn, you may miss something valuable if it lies further down your decision tree and resting makes that opportunity more accessible. This is certainly normal, but isn't unique to impartial altruism.

We shouldn't always prefer learning to doing nothing—it's even possible that we should do it less often than some other things. This is because successful approaches to epistemic improvement should mix learning with other things, such as resting when tired, eating when hungry, not dying in the process, etc. But isn't it possible for us to identify some combination of such low-impact actions that is better than, for instance, indefinitely pursuing just one of them? Empirically, this seems possible in many domains, and we may be able to identify some similarities across them. So again, it seems unclear, what is it about impartial altruism that makes things more clueless here—to the degree that we can't even decide whether we better be alive and learn things.

It will be helpful to have greater clarity about the potential sources and mechanisms of cluelessness (and their relative strength) in such specific most-absurdly-looking cases.^[6]

DiGiovanni states:

We aren’t justified in c-preferring any action over any other

A single counterexample of a pair of actions where one is c-preferred over the other would suffice to disprove his claim.

It seems noteworthy that none of the solutions attempted this.

Not sure whether new posts count for the commentary competition, but this seemed too long to paste here as a literal comment!

The maximality rule is too demanding. Under maximality, we can't even say that [1, 10 000] is better than [-10 000, 1.0001]. Are there any good reasons to avoid more permissive rules even in cases like this?

Of course, if your UEV intervals overlap and you choose one of them, you may make a mistake if your idealized version would choose the other one. But choosing at random is no better in this regard.

What seems important is how costly such mistakes are, i.e., how the overall performance of your choice rule compares with that of other rules—such as choosing at random.

Can these performance measures be defined without additional assumptions about how values are distributed within UEV intervals?

Just sharing my quick takes on how to navigate cluelessness with a rather long-term strategy here. It doesn't get to predictability land but to enough insight to move forward carefully despite vast uncertainties. 

Reading through all the essays and comments this week, I've become much more convinced that cluelessness is pervasive and standard ways of handling it fail. I agree with the normative and conceptual premises.

But the empirical premise is untrue. In particular, it is untrue that for any pair of actions, the actions' consequences are too coarse to be compared.

Consider the following counterexample: I could either donate 10% of my income to GiveDirectly or take a nice vacation. Is it true that our knowledge of the consequences of these actions is so coarse-grained that the comparison between them is indeterminate? I don't think so, even though in principle the same concerns about the catch-all and unawareness apply.

The "best guess" justification for donating is simple. My lifetime income is very likely to be in the top 5% globally; under all plausible theories of utility, marginal utility declines with income; thus, the marginal transfer from me to someone in extreme poverty improves aggregate well-being.

How might this be incorrect or imprecise?

• The recipient of the donation could use it for something bad. (This is analogous to the meat-eater problem, but given my own consumption of animal products the meat-eater problem itself doesn't apply.)
• Catch-up growth in recipient countries could lead to an unstable multi-polar world order, which eventually causes a catastrophic WWIII.
• Because I don't go on vacation, I don't accidentally step on a butterfly I otherwise would have. The butterfly flaps its wings and causes a tornado that kills 50 people. Also, those 50 people were all wild animal suffering researchers.
• Everyone else is a p-zombie; only my consumption matters, and I should take more vacations.

In case this seems too glib, I promise I tried to come up with better examples (and asked AI). I do not think there is any plausible probability distribution under which the above are likely enough that taking the vacation is better than donating. I would welcome suggestions.

"Plausible" is doing a lot of work here. I don't have a good justification for why no probability distribution where these events are likely enough to change the outcome of the comparison is plausible to me. All I can say is that if we consider these type of events to be equally plausible to the "best guess", then I agree with Richard Y Chappell that cluelessness considerations approach radical skepticism.

More formally, I would claim that the set S of all possible actions contains a non-empty proper subset C of actions over which an ideal impartially altruistic agent has complete preferences. The comparison between each action in C and each action in S\C is indeterminate. This still offers a lot of guidance on impartial altruistic action, because I would claim we all have more than one action within C available to us.

There is still a lot of debate to be had and research to be done to figure out which actions are in C and which aren't (see Jim Buhler's "The Train to Arbitrary Land" for a good elaboration of this problem.) 

Still, this offers a meaningful change from previous cause prioritization discussions, which generally assumed that all actions are in C. Practically, these considerations push me towards near-term interventions, whose sign is clear and comparisons of which with alternative uses of resources I believe to be determinate.

The problem of Cluelessness has been grappled with for millennia; two relevant examples:

1. The story of Khidr in the Quran, who, through a series of nonsensical or seemingly evil actions, demonstrates to Moses how we are Clueless about consequences.

2. The Bhagavad Gita, which advises disregarding consequences altogether:

To action alone hast thou a right and never at all to its fruits; let not the fruits of action be thy motive; neither let there be in thee any attachment to inaction."

The solutions end up looking like Deontology or Virtue Ethics, which Anthony (inadvertently?) references in his summary: "Other values and moral norms still matter to us, for example, rules like avoiding dishonesty or virtues like compassion."

Anyone grappling with the possibility of Cluelessness would do well to consider prior art.

For the sake of provocation: How is managing Cluelessness in Philanthropy any different from managing Cluelessness in, say, running a neighborhood coffee shop? 

There is an unknowable amount of cause-effect relationships involved in running a business. Is there anything unique about Philanthropic modeling? Modeling dynamical systems is, in general, notoriously hard. 

I wrote a few paragraphs fleshing out my question here if anybody wants to respond to me more in full.