I’m one of the judges of the competition. My comments shouldn't be taken as a full review of a post. And, unfortunately, I won’t have capacity to comment on every post or engage with all replies. Thanks so much to everyone who entered!
I find the case for bottom-up bracketing very well-argued and compelling.
I’d like to see more engagement with non-identity issues that threaten the action-guidingness of bottom-up bracketing.
Suppose a not-yet-existent Alice will exist with a happy life in worlds where I do A and does not exist in worlds where I do B. If Alice’s welfare is zero in B-worlds, then she prefers A over B, and so enters the aggregation. But aggregating over all such future beings presumably leaves the aggregate comparison indeterminate (e.g., maybe these are all the beings that will live in a utopia if x-risk is averted, and we’re clueless about how A and B affect x-risk).
Instead of setting Alice’s welfare to zero in worlds where she doesn’t exist, we could use some kind of counterpart relation that ends up making Alice’s comparisons between worlds in which she does and doesn’t exist indeterminate (e.g., Michael’s comment here). Since the contingent persons would then all be bracketed out, you might actually end up with a determinate comparison that way. But such views seem in tension with the post’s person-centred perspective.
I’m one of the judges of the competition. My comments shouldn't be taken as a full review of a post. And, unfortunately, I won’t have capacity to comment on every post or engage with all replies. Thanks so much to everyone who entered!
I think this is a neat and thought-provoking argument that ought to give proponents of cluelessness pause.
My thoughts on this haven’t fully settled, but some reactions:
Suppose that, per the post’s argument, the most choiceworthy actions according to (the most plausible version of) imprecise consequentialism are the somewhere-uniquely-optimal ones. There is still a question of how this view should enter into intertheoretic aggregation. I think it would be odd for imprecise consequentialism to dominate the intertheoretic aggregation, given big disagreement between different elements of the representor about the value of somewhere-uniquely-optimal actions.
Example:
A is great according to common-sense morality, and is nowhere-optimal.
B and C are meh according to common-sense morality, and are uniquely optimal according to p_B and p_C, respectively. Also, B is terrible according to p_C and C is terrible according to p_B.
It seems to me that A should at least be permissible, when aggregating over these two theories.
I’m not sure yet that we can distinguish somewhere-uniquely-optimal and nowhere-uniquely-optimal actions. The space of policies available to us in real life is extremely high-dimensional and it’s not yet clear to me that you couldn’t run a similar argument to your MAWF example to cast doubt on the somewhere-optimality of any policy.
I tend to think that one should never be required to strictly c-prefer a mixture of A and B to C whenever neither A nor B is c-preferred to C. (For one thing, this involves a kind of dynamic inconsistency: I know that, once the coin lands, no matter how it lands, I will no longer have a c-preference between the continuation plan and C.) I tend to think that it’s an unfortunate feature of the dominance rule that this happens, and I would love it if there was some repair to the dominance rule that didn't have this property. Sadly I don’t have one to give, and I suspect any such theory would have its own significant costs.
But, if there were a way to avoid the conclusion that a nowhere-optimal action is always c-dispreferred to some mixture, then there is only the weaker argument that we ought not choose nowhere-optimal actions. And I’m not yet convinced that nowhere-optimality is a problem.
In principle the proposal in that post is supposed to encompass a larger set of bracketing-ish things than the proposal in this post, e.g., bracketing out reasons that are qualitatively weaker in some sense. But the latter kind of thing isn't properly worked out.
Good question! Yeah, I can’t think of a real-world process about which I’d want to have maximally imprecise beliefs. (The point of choosing a “demon” in the example is that we would have good reason to worry the process is adversarial if we’re talking about a demon…)
(Is this supposed to be part of an argument against imprecision in general / sufficient imprecision to imply consequentialist cluelessness? Because I don’t think you need anywhere near maximally imprecise beliefs for that. The examples in the paper just use the range [0,1] for simplicity.)
cluelessness about some effects (like those in the far future) doesn’t override the obligations given to us by the benefits we’re not clueless about, such as the immediate benefits of our donations to the global poor
I don't think that's unreasonable. Personally, I strongly have the intuition expressed in that quote, though definitely not certain that I will endorse it on reflection.
Wouldn't the better response be to find things we aren't clueless about
The background assumption in this post is that there are no such interventions.
> We start from a place of cluelessness about the effects of our actions on aggregate, cosmos-wide value. Our uncertainty is so deep that we can’t even say whether we expect one action to be better than, worse than, or just as good as another, in terms of its effects on aggregate utility. (See Section 2 of the paper and resources here for arguments as to why we ought to regard ourselves as such.)
It depends on what the X is. In most real-world cases I don’t think our imprecision ought to be that extreme. (It will also be vague, not “[0,1]” or “(0.01, 0.99)” but, “eh, seems like lots of different precise beliefs are defensible as long as they’re not super close to 1 or 0”, and in that state it will feel reasonable to say that we should strictly prefer such an extreme bet.)
But FWIW I do think there are hypothetical cases where incomparability looks correct. Suppose a demon appears to me and says “The F of every X is between 0 and 1. What’s the probability that the F of the next X is less than ½?” I have no clue what X and F mean. In particular, I have no idea if F is in “natural” units that would compel me to put a uniform prior over F-values. Why not a uniform prior over F^2 or F^-100? So it does seem sensible to have maximally imprecise beliefs here, and to say it’s indeterminate whether we should take bets like yours.
Yes, it feels bad not to strictly prefer a bet which pays 10^10 if F < ½. But adopting a precise prior would commit me to turning down other bets that look extremely good on other arbitrarily-chosen priors, which also feels bad.
Some reasons why animal welfare work seems better:
I put some weight on a view which says: “When doing consequentialist decision-making, we should set the net weight of the reasons we have no idea how to weigh up (e.g., long-run flowthrough effects) to zero.” This probably implies restricting attention to near-term consequences, and animal welfare interventions seem best for that. (I just made a post that discusses this approach to decision-making.)
I think this kind of view is hard to make theoretically satisfying, but it does a good enough job of capturing intuitions relative to alternatives that I currently want to give it some weight.
Non-consequentialist considerations might push towards fighting the worst ongoing atrocities / injustices, which also suggests animal-related work.
Thanks for this! IMO thinking about what it even means to do good under extreme uncertainty is still underrated.
I don’t see how this post addresses the concern about cluelessness, though.
My problem with the construction analogy is: Our situation is more like, whenever we place a brick we might also be knocking bricks out of other parts of the house. Or placing them in ways that preclude good building later. So we don’t know if we’re actually contributing to the construction of the house on net.
On your takeaway at the bottom, it seems to be: “if someone doing A is a necessary condition for a particular good outcome X, that’s a reason for you to do A”. Granted. But the whole problem is that I don’t know how to weigh this reason against the reasons favoring me doing not-A. Why do you think we ought to privilege the particular reason that you point to?
We at CLR are now using a different definition of s-risks.
New definition:
S-risks are risks of events that bring about suffering in cosmically significant amounts. By “significant”, we mean significant relative to expected future suffering.
Note that it may turn out that the amount of suffering that we can influence is dwarfed by suffering that we can’t influence. By “expectation of suffering in the future” we mean “expectation of action-relevant suffering in the future”.
I’m one of the judges of the competition. My comments shouldn't be taken as a full review of a post. And, unfortunately, I won’t have capacity to comment on every post or engage with all replies. Thanks so much to everyone who entered!
I find the case for bottom-up bracketing very well-argued and compelling.
I’d like to see more engagement with non-identity issues that threaten the action-guidingness of bottom-up bracketing.
I’m one of the judges of the competition. My comments shouldn't be taken as a full review of a post. And, unfortunately, I won’t have capacity to comment on every post or engage with all replies. Thanks so much to everyone who entered!
I think this is a neat and thought-provoking argument that ought to give proponents of cluelessness pause.
My thoughts on this haven’t fully settled, but some reactions:
In principle the proposal in that post is supposed to encompass a larger set of bracketing-ish things than the proposal in this post, e.g., bracketing out reasons that are qualitatively weaker in some sense. But the latter kind of thing isn't properly worked out.
Good question! Yeah, I can’t think of a real-world process about which I’d want to have maximally imprecise beliefs. (The point of choosing a “demon” in the example is that we would have good reason to worry the process is adversarial if we’re talking about a demon…)
(Is this supposed to be part of an argument against imprecision in general / sufficient imprecision to imply consequentialist cluelessness? Because I don’t think you need anywhere near maximally imprecise beliefs for that. The examples in the paper just use the range [0,1] for simplicity.)
It sounds like you reject this kind of thinking:
I don't think that's unreasonable. Personally, I strongly have the intuition expressed in that quote, though definitely not certain that I will endorse it on reflection.
The background assumption in this post is that there are no such interventions.
> We start from a place of cluelessness about the effects of our actions on aggregate, cosmos-wide value. Our uncertainty is so deep that we can’t even say whether we expect one action to be better than, worse than, or just as good as another, in terms of its effects on aggregate utility. (See Section 2 of the paper and resources here for arguments as to why we ought to regard ourselves as such.)
Thanks, Ben!
It depends on what the X is. In most real-world cases I don’t think our imprecision ought to be that extreme. (It will also be vague, not “[0,1]” or “(0.01, 0.99)” but, “eh, seems like lots of different precise beliefs are defensible as long as they’re not super close to 1 or 0”, and in that state it will feel reasonable to say that we should strictly prefer such an extreme bet.)
But FWIW I do think there are hypothetical cases where incomparability looks correct. Suppose a demon appears to me and says “The F of every X is between 0 and 1. What’s the probability that the F of the next X is less than ½?” I have no clue what X and F mean. In particular, I have no idea if F is in “natural” units that would compel me to put a uniform prior over F-values. Why not a uniform prior over F^2 or F^-100? So it does seem sensible to have maximally imprecise beliefs here, and to say it’s indeterminate whether we should take bets like yours.
Yes, it feels bad not to strictly prefer a bet which pays 10^10 if F < ½. But adopting a precise prior would commit me to turning down other bets that look extremely good on other arbitrarily-chosen priors, which also feels bad.
Some reasons why animal welfare work seems better:
(Thanks! Haven't forgotten about this, will try to respond soon.)
Thanks for this! IMO thinking about what it even means to do good under extreme uncertainty is still underrated.
I don’t see how this post addresses the concern about cluelessness, though.
My problem with the construction analogy is: Our situation is more like, whenever we place a brick we might also be knocking bricks out of other parts of the house. Or placing them in ways that preclude good building later. So we don’t know if we’re actually contributing to the construction of the house on net.
On your takeaway at the bottom, it seems to be: “if someone doing A is a necessary condition for a particular good outcome X, that’s a reason for you to do A”. Granted. But the whole problem is that I don’t know how to weigh this reason against the reasons favoring me doing not-A. Why do you think we ought to privilege the particular reason that you point to?
We at CLR are now using a different definition of s-risks.
New definition:
S-risks are risks of events that bring about suffering in cosmically significant amounts. By “significant”, we mean significant relative to expected future suffering.
Note that it may turn out that the amount of suffering that we can influence is dwarfed by suffering that we can’t influence. By “expectation of suffering in the future” we mean “expectation of action-relevant suffering in the future”.