Comments
When I announced April’s Better Futures Highlight Week,[1] I confidently wrote: “To make the future go better, we can either work to avoid near-term catastrophes like human extinction or improve the futures where we survive.”
But what if I was wrong? What if both options are bankrupt, and we have no justified answer as to whether any particular work would make the future better? In his Challenge of Unawareness sequence, Anthony DiGiovanni argues that we are in just this position.
Specifically, he argues that our evidence about the future is so poor that our assessments of impact (explicit expected-value estimates and informal best guesses alike) should be severely indeterminate, yielding no verdict on which actions better serve the impartial good.
If Anthony is correct, the implications for EAs are severe.[2] Yet there has been little public response, and no definitive critique. With this contest, we hope to surface new solutions for the altruist who buys Anthony’s conclusion, and/or new reasons to think that we aren’t as clueless as Anthony claims. More details below.
By August 14th, 11:59pm anywhere on earth: Submit your entry via this form. You can message Toby with questions while you are drafting—including clarification questions about the sequence itself/ what you are arguing.
You may submit multiple entries as long as they are substantively different from each other. More eligibility information is below.
During the week August 17–23: Posts eligible for judging will be posted on the EA Forum, and the comment competition will take place.
Week of August 24th: Winners of the post competition and the comment competition will be announced.
Every eligible entry does one of the following three things: challenge a premise, challenge the inference, or offer a constructive response to Anthony's sequence. On the submission form you’ll have to select which.
Above all, we want focused entries: the most useful submissions zero in on a single, clearly-identified step of the argument rather than taking on the whole thing at once. Anthony’s summary post breaks the argument into the named premises outlined below and tabulates the main existing counterarguments to each—read it first, and use it to orient your entry.
Whichever route you take, your entry will only be published on the EA Forum and judged if it shows a clear grasp of Anthony’s argument. Posts can be whatever length they need to be: the winner could be a concise 200 words or a paper-length 5000.
Argue that one specific premise is false or unmotivated. You must say which one you’re targeting:
Alternatively, go finer-grained: take one of the specific counterarguments Anthony lists in his table under a given premise, and explain why his response to it fails.
Grant all the premises but argue that the conclusion—that we have no impartial-altruistic reason to prefer any action over another—doesn’t follow.
Accept the argument—and tell us what the impartial altruist should do anyway. Grant the conclusion, but offer a constructive response: defend a solution Anthony considers and rejects (e.g. wagering), or propose something entirely original.[3]
Macroscopic Ventures is offering prizes of $2500, $1500 and $1000 for the three best eligible[4] posts.
Outlier Prize
Beyond the prizes above, we’re offering a much larger reward for a truly exceptional entry—one that meaningfully changes how we think about cluelessness. If the judges consider your contribution that significant, Macroscopic will award up to $50,000. Two kinds of entry could clear the bar:
This is a very high bar, and we’ll be very pleasantly surprised if we get to award this prize[5]—but we want to signal how much we’d value a genuine breakthrough. And even short of the bar: if your entry sketches a promising idea, Macroscopic may fund you to develop it further, with the outlier prize still on the table should that work mature into something exceptional.
Referral Prize
Perhaps you yourself don’t want to enter, but you know a friend who might. If you refer said friend and they win one of the prizes with you listed as their referrer in their submission form, you yourself will win $250.
Apologies in advance if you have fifteen friends reaching out to you. Please only mention the first referrer on your form.
During the event week (August 17–23), there will be discussion of the competition entries, and of the unawareness sequence itself. We’d like to incentivise high-quality discussion, so we’re running a secondary competition, during the week, for the best comments. Up to $2000 in comment prizes will be awarded—more details nearer the time.
On top of what we explain above in the ‘What your entry should look like’ section, the judges will be looking for the following:
Arguments put forward in this contest could in principle influence Macroscopic’s funding priorities (though the bar for doing so is high). Furthermore, Macroscopic may consider strong contestants for longer-term funding to research cluelessness and/or adjacent topics.
This list may change a little before judging begins; I’ll make edits on this post if and when it does. Judges are not eligible for cash prizes.
To enter, you must be 18 or over and not a judge or member of Macroscopic Ventures staff.[8] Judges’ decisions are final and at their sole discretion.
You’re welcome to enter anonymously or under a pseudonym, but to receive a prize you’ll need to provide your legal name, address, payment details, and any documents Macroscopic reasonably ask for to complete the necessary checks. Prizes cannot be issued to people located in jurisdictions subject to comprehensive sanctions. Unfortunately, a prize cannot be paid if the necessary checks can’t be completed.
Prizes will be paid in USD within about 2 weeks of selection and completion of checks. Winners are responsible for any taxes in their own jurisdiction; Macroscopic Ventures and the contest organizers and judges don’t provide tax advice.
Same rules as the Essays on Longtermism competition. You are permitted to use AI in whatever way you think best; however, relying on AI-written text in your final submission is extremely likely to render you ineligible (because the argument will be bad, the AI won’t grok the sequence, etc...).
We do nonetheless care about the quality, and therefore the usefulness, of the submissions above all else. If, somehow, the best submission is AI generated, it will win first place.
Yes! The prize (if you win) will be split evenly between the co-authors.
Not for this competition. No pieces which have been published (including blog posts) before the date of this announcement post will be considered. This is because we care about incentivising new content, rather than surfacing the best (as we did for the creative writing competition).
However, once the August 14th deadline has passed, you are very welcome to post your entry anywhere you please (yes, even before judging is completed).
We can’t promise feedback on posts. However,
You won’t be marked on style, but readability is always a plus (also, I stand by this writing advice). Additionally, though you are very much encouraged to disagree with the arguments in the sequence, being too polemical is likely to get in the way of a good argument.
-
Have we missed anything? Ask your questions about the competition in the comments below.
At first glance, one could be forgiven for thinking the implication is to veer into nihilism. However, Anthony outlines in his sequence what other responses might look like: see this section in particular.
Bracketing is one attempt at an original, constructive response.
If there are less than three posts deemed eligible (by the contest organizers or judges), fewer prizes will be given out. More on eligibility in What your entry should look like.
If multiple entries meet this bar, we will award multiple outlier prizes.
And the author of Maximal Cluelessness. See here for Andreas’ full list of publications.
Staff and judges can still write posts and comment – indeed they are encouraged to – they just can’t win the money.
I'm SUPER excited for this one. I find trying to think sanely about cluelessness hard and a bit demoralizing at times (and I've heard that from others too). I'd really value reading more about ways people who've spent more time on this orient to the topic.
I'll be following along!
I don't know how to challenge the premises because the key premise seems to be an assertion that I don't find convincing.
In 'Should you go with your best guess?', which appears to be the primary argument against the idea of 'deterministic' Bayesian credences, DiGiovanni repeatedly signposts that he's going to give an argument against them... but I can't see anything that constitutes one.
In the section 'Background on degrees of belief and what makes them rational', he talks about how we don't get to find out which beliefs outperform others, but doesn't say why this means we shouldn't/can't pick credences according to our best effort. It also doesn't say why, if we can measure short term value, we shouldn't use that as a justification for our decisionmaking process and assume EV from events that we don't think we can assess is 0.
In the section 'Motivating example', he gives an example vignette, at the end of which we're given that 'this feels so arbitrary'. But it doesn't seem like 'feeling arbitrary' is a reason not to do something - especially when we're not given an alternative (or at least, no other decision process that seems less or equally arbitrary).
So my response is just to say 'using credences still seems fine, if occasionally emotionally uncomfortable (maybe using distributions is sometimes empirically better, and if so I support it)' - in which case I don't see a problem in need of solving.
Hi Arepo, thanks for sharing your cruxes here.
The argument I give against assigning precise credences is that it's arbitrary — literally, you pick one precise credence over many others for no reason. To me, "you have no reason to do this thing" is a pretty darn strong argument. :) (ETA: I like the intuition pump in this very short post, if it helps.)
doesn't say why this means we shouldn't/can't pick credences according to our best effort.
Why does "our best effort" need to be precise? Can you say more what exactly you mean? (If the intuition is that more precision = more information, I address that in the post.)
It also doesn't say why, if we can measure short term value, we shouldn't use that as a justification for our decisionmaking process and assume EV from events that we don't think we can assess is 0.
I address this in the unawareness sequence. I recommend reading the table in my summary post — the row with "Even if our impact is dominated by consequences we’re unaware of..." — for the high-level idea, and the links therein for details.
especially when we're not given an alternative
Isn't this privileging the hypothesis? My claim is that we don't have a positive argument in favor of doing what the precise EV approach recommends (or fuzzier "best guesses", either). If our best defense of that approach is "what else is there?", that seems rather damning.
literally, you pick one precise credence over many others for no reason
At some foundational level, a credence has no deeper reason than 'some neurons fired that way'. But you don't need to restrict yourself to concerns about the whole future lightcone to run into this problem - at the foundational level this is true of every statement.
There are various ways one might respond to this challenge, but if we don't view it as insurmountable elsewhere, I don't see why we should do so with credences (which are of course usually non-foundational statements). And if we do, it undermines e.g. any argument about unawareness.
Why does "our best effort" need to be precise?
It only needs to be as precise as is necessary for decision-making. I will probably never need to forecast rain to 8 decimal places. But if you're saying forecasting rain as 'less than .5' is ok, but that forecasting 0.1234567% chance of rain if the extra precision was actually decision-relevant, would be un-ok/impossible/qualitatively different, then I disagree.
(from link) It doesn’t follow from “we don’t know the net direction of the consequences we’re unaware of” that we should regard the positives and negatives as precisely symmetric. One reason symmetry is implausible: If we become aware of a new possible consequence, this should update our beliefs about the others we’re unaware of, breaking the symmetry.
If you 'become aware' of something, you've gained information and should update your priors accordingly. That doesn't move me away from being happy to treat genuine unknowns as EV-0. Your counterpoint seems to be that in some cases that feel sort-of- equal (and about which, in the cases you describe we actually have a lot of information), we might be inclined to give equal credence. But it seems to me correct to say 'if you have meaningful knowledge of two possible outcomes, and the weight you assign to them is decision-relevant, giving them equal credence is a mistake', which fixes this purported problem without radically undermining our epistemology.
My claim is that we don't have a positive argument in favor of doing what the precise EV approach recommends
The precise EV approach is well evidenced in short-term decision-making, so the positive argument is that there isn't any principled difference between short and long-term decision-making
(Due to time constraints I expect I can only give brief replies/clarifications, going forward. I hope a full read of the sequence will suffice, though I realize it's quite long, sorry!)
But you don't need to restrict yourself to concerns about the whole future lightcone to run into this problem - at the foundational level this is true of every statement. ... I don't see why we should do so with credences (which are of course usually non-foundational statements)
(See my last para for the "future lightcone" thing.)
I don't understand your Munchausen trilemma argument yet. You say credences are "of course usually non-foundational". Agreed! That's exactly why I think our choices of credences require deeper justification. (Whereas foundational things, like Huemer's "seemings", don't.[1])
forecasting 0.1234567% chance of rain if the extra precision was actually decision-relevant
The extra precision might be "decision-relevant" in the sense that: if you were justified in a credence of 0.1234567% + 0.0000001%, you should choose A, and if you were justified in a credence of 0.1234567% - 0.0000001%, you should choose B. But the whole question is why we'd be justified in the former vs. the latter, epistemically. ("I need to make a choice" isn't a justification for any particular option you choose.)
Your counterpoint seems to be that in some cases that feel sort-of- equal (and about which, in the cases you describe we actually have a lot of information), we might be inclined to give equal credence.
That's not what I'm saying, sorry — I'm denying we should give equal credence. Please see my reply to a similar comment here, and section 3.2.1 and 4.1.1 of the sequence (you might need to CTRL+F some terms defined earlier in the sequence). If it's still unclear, I'm happy to try to explain further if you could point to particular passages that need clarification.
The precise EV approach is well evidenced in short-term decision-making
I don't know what exactly this means. If you mean "we seem to be justified in using precise EVs in short term decision making":
That is, they don't require deeper justification prima facie. They're still defeasible.
I feel somewhat confused about what exactly the challenge here is. You say:
Grant all the premises but argue that the conclusion—that we have no impartial-altruistic reason to prefer any action over another—doesn’t follow.
My understanding is that Anthony agrees we can justifiably prefer actions, and even do so on altruistic consequence-based grounds if something like bracketing works — but he'd deny that either deontological reasons or bracketed reasons are "impartial altruistic reasons" in the sense his conclusion targets: the first aren't consequence-comparisons at all, and the second are consequence-comparisons that give up full impartiality to stay determinate.
Is my understanding correct? If so, I would find a more precise statement of the inference people are supposed to challenge helpful.
Hey Ben — your understanding is correct. Option 2 is meant to allow for the possibility that the inference from P1-P3 to Conclusion, as stated in my summary post, is logically invalid. (Of course I think that's unlikely, but philosophy can be subtle.) Does that clear things up?
Would you say bracketing isn't fully impartial? It doesn’t seem less impartial than Scanlon's Greater Burden Principle (or Tom Regan's harm principle), which roughly says to choose between two actions, you should prioritize the individual(s) with the strongest claim or who has the most to lose counterfactually between options (not necessarily the worst off, to contrast with Rawls' Difference Principle/maximin/leximin).
I guess methods of bracketing where admissible bracketings must group certain individuals, e.g. spatiotemporally or by causal paths affecting them, are more partial. But if you go with the unconstrained version, it seems basically impartial?
It is sensitive to differences between worlds beyond the number of individuals at each welfare level, because it has to identify each individual across worlds to take individual differences.
Hmm yeah maybe I shouldn't have fully endorsed Ben's summary. I think many forms of bracketing are impartial in the sense that they don't arbitrarily favor some moral patients over others. But the forms of bracketing I'm aware of are either (1) not "impartial" in the sense that some moral patients/consequences are bracketed out for not very well-motivated reasons, or (2) not action-guiding.
That latter definition of impartial might be confusing though, so in general I'd just list my specific dissatisfactions with different forms of bracketing, "impartiality" aside.
(1) not "impartial" in the sense that some moral patients/consequences are bracketed out for not very well-motivated reasons, or (2) not action-guiding.
If you think it's fair to map the "same" moral patient across worlds to themselves, assuming at least some transworld identity, then I'd guess we could work out counterpart relations extending transworld identity to cover contingent moral patients, maybe along the lines of Thomas, 2019, pp.30-31 where, roughly, we map each contingent moral patient under one choice to the statistical average of the contingent moral patients — across all of spacetime and the whole multiverse if we're in one — under the other choice.[1]
At first pass, if we're clueless about the total welfare effects for any action or comparing any two actions, I'd guess we'd be clueless about the average contingent moral patient, and, at least on bottom-up bracketing, just bracket them out and ignore them. This would mean the theory acts like the strict person-affecting view, where only currently existing (or at least "necessary") moral patients count, and we ignore (at least) the rest. That might sound bad, but we ignore them because we're consistently clueless about whether their counterparts are better or worse off in expectation, which seems like a good principled reason to do that.
This could be practically action-guiding. You'd probably focus on near-term human welfare, because currently existing non-human animals that you could "affect" cost-effectively don't typically live long enough for your actions today to help them (e.g. most live <2 years before slaughter, often <1 year), and you're clueless about the welfare of the counterparts of the future/contingent animals you could "affect" cost-effectively.
But maybe speculative scenarios, including how TAI might play out, also make you clueless about how to benefit currently existing humans if, before bracketing, you're aggregating and taking expectations over their entire futures or their current preferences pointing at things at least a few years away, and can't bracket out the speculative scenarios. I imagine you could get around this by using transworld identity at each point of time[2] and do bracketing before aggregating over their futures, so you can bracket out the periods of time you're clueless about.
Very interesting to see but isn't it a little strange that the jury is comprised of the author of the work to be criticised and their co-author? Doesn't that risk muddling their incentives when judging the contest entries?
Thanks for raising this. I agree the setup isn’t ideal, but the listed judges are the people who combine sufficiently good judgement + relevant expertise and—crucially—willingness to put in the time. (I did try to enlist other experts, but it didn’t come together.)
the jury is comprised of the author of the work to be criticised and their co-author
To clarify, the essay panel isn’t just the author plus a collaborator—it’s three people: Anthony (who wrote the sequence), Jesse (who collaborated on related work), and Andreas Mogensen, an Oxford philosopher who has previously worked on cluelessness but who had no involvement in the sequence and is independent of it. I’m not sure this fully dissolves the biases/incentives concern, but I hope it goes some way toward doing so.
Thank you for this! I actually had myself had some thoughts much akin to Anthony DiGiovanni's and also satisfied myself with my own version of bracketing (I referred to it in my mind as "local impact" as opposed to the cosmic-scale impact of which we can not really know anything about) without having any idea that others had had similar thoughts and had even given them names!! XD It's so satisfying to know others have grappled with similar doubts!
Here's my attempt at Option 3:
My justification for using bracketing (i.e. discounting cosmic-level uncertainty about all the possible ramifications and externalities of my actions) is that I am ultimately self-interested in wanting to do 'altruism': I am only 'altruistic' because it feels genuinely nicer to *me* to live in a world where others are healthy/happy/flourishing compared to one where they're not. Therefore, I shouldn't concern myself with consequences or externalities that are too much outside of my immediate field of perception / conceptualisation.
Does that make me a bad """altruist""" in absolute terms? Probably. But practically:
I am comfortable in my cluelessness when trying to do good because all of my attempts to do good actually end up being a net positive for me in terms of my own self-interest and in the worst case scenario I am either happily and innocently ignorant of my evilness (not great but oh well) or my 'negative' actions will be endlessly viewed either as positive or negative depending on the cosmic viewpoint.
PS: I also enjoy the intellectual challenge of finding effective solutions to what, from my limited cosmic vantage point as a human alive on Earth in the 21st century, consider to be 'problems'. Another point in terms of self-interested reasons to do
PPS: if somebody enjoyed reading this and is interested in how the problem of cluelessness when trying to do good is treated in ancient texts, I'm pretty convinced that Arjuna's dilemma in the Bhagavad Gita is about this (he is about to basically fight his cousins to death on 'moral' grounds and wonders whether this is the actual right action, and has a dialogue with Krishna about the 'art' (?) and nature of right actions.)
My understanding is that Anthony agrees that there are still reasons to do things:
First, the unawareness argument doesn’t imply that “nothing we do matters” all things considered. It only implies that impartial altruism, or any very far-reaching value system, isn’t action-guiding. Other values and moral norms still matter to us, for example, rules like avoiding dishonesty or virtues like compassion. These can be action-guiding even if we’re clueless about total consequences.
I think your justification "because all of my attempts to do good actually end up being a net positive for me in terms of my own self-interest" doesn't disagree with his conclusion?
Thanks Ben!
As I said, I'm accepting the argument (that's why I chose option 3!).
Option 3: Offer a constructive response
Accept the argument—and tell us what the impartial altruist should do anyway. Grant the conclusion, but offer a constructive response: defend a solution Anthony considers and rejects (e.g. wagering), or propose something entirely original.[3]I
I indeed can't find any impartial-altruistic rational reasons to pick one course of action over another if I reason at a cosmic scale, but my response explained why people should do effective altruism anyway (self-interest)! 😁
NB: I only read the summary by Toby and didn't engage with the larger body of posts from Anthony and others, so I most probably am missing crucial nuance TBF!
About the small portion of text you quoted (could you please let me know where it's from btw? EDIT: Just found it on your link, will read full text later!): I'm not sure I understand that compassion or avoiding dishonesty are different from Effective Altruism. To me the latter is the same as the formers but applied at a slightly different scale (e.g. Effective Altruism advocates being compassionate about all humans on the planet equally instead of say, preferring to show more compassion to those who are local to us). Therefore I also don't understand the point about compassion or avoiding dishonesty being able to act as action-guiding principles Vs EA not being able to. Finally I don't really understand what 'action-guiding' means as a concept - from your text it seems it has a precise definition which I'd love to learn more about.
I have a clarification question regarding the judges' interpretation of "constructive" response. For the question to make sense, though, I need to offer a bit of context.
One way to understand the meaning of "constructive response" is to construe it as "a response that gets the 'impartial altruist' out of a perceived pickle, allowing them to continue to inhabit the 'impartial altruist' orientation in some form."
The pickle is "Yipes! If we're really clueless, then this EA stuff maybe isn't so E after all." The constructive response is "Well, maybe it's still as E as we can hope for, given our station, relative to the other options". Or something.
As shorthand, we can think of this kind of constructive response as an "internally constructive" response: inside the orientation of the 'impartial altruist,' it constructs an escape hatch from the pickle that allows one to retain at least some level of confidence that sticking with some version of the 'impartial altruist' orientation is advisable.
Another way that a response might be "constructive" is by liberating one from an orientation that is no longer advisable to retain. Such a response is constructive not because it builds an internal escape hatch from the pickle, but because it constructs an escape hatch from the orientation itself.
As shorthand, we can think of this kind of constructive response as an "externally constructive" response: given that it is generally a good (constructive) thing to escape a failing enterprise, the former 'impartial altruist' sees, from outside their past orientation, the opportunity costs of staying within it and the prospects of moving beyond it.
In my field, philosophy, one might see W.V.O. Quine's "Two Dogmas of Empiricism" as an externally constructive response to logical positivism, or Saul Kripke's Naming and Necessity as an externally constructive response to ordinary language philosophy, or Wilfird Sellars' Empiricism and the The Philosophy of Mind as an externally constructive response to naive foundationalism in epistemology.
Maybe it's more descriptively accurate to call externally constructive responses, as I'm thinking of them, as "program ending" responses, but it does seem to me that they are constructive in an important sense given that abandoning flagging and failed prospects can save a lot of time and energy that can be more wisely repurposed.
My question is whether judges would consider "externally constructive"/"program ending" contributions eligible to win prizes. I'm curious about this because I have conflicting intuitions.
On the one hand, I can't imagine that the (fictional) Effective Logical Positivism Society would have ever considered awarding W.V.O Quine a 1,325 pound prize (roughly $50,000 in 1951 BPS) had he submitted "Two Dogmas of Empiricism" to a similar essay contest. In a similar vein, one wouldn't expect a (fictional) Christian Theodicy Society to award a prize to an essay that persuasively shows theodicy to be a flawed enterprise. When an organization is promoting a particular paradigm, program, or identity, it is counterintuitive that they would fete an essay that undercuts the paradigm, program, or identity they support.
On the other hand, EA's commitment to impartiality and to following the evidence where it leads, even if it leads to counterintuitive or unwelcome conclusions, invites one to wonder whether (what I'm calling) an "externally constructive"/"program ending" essay wouldn't be very desirable for its allowing one to put aside something that is underperforming or failing so that the resources going into it can be reinvested more wisely without nostalgia for the loss of a certain favored identity or method.
Good grief! I had intended for this to be three or four sentences! Apologies for the length!
Beautifully put, at least from my perspective :) You've articulated something I've been struggling to find the right words for.
Hey Matthew!
We'd be happy with externally constructive critiques. For example, there are responses we would value which argue that we should not aim to be impartial altruists or that we should let go of even the broad conception of 'expected value' reasoning from Premise 1.
However, eligible posts for the competition will respond to one of the premises, defend a counterargument, or pose a constructive resolution. If your argument doesn't fall into one of these, I'd urge you to post it during the event week anyway (August 17th) and take part in the discussion, but it wouldn't be eligible for the competition.
A judge also said to me: "I'd think that 'externally constructive' responses are OK, but good ones will answer this problem: It seems like any plausible moral view is going to require some limited form of consequentialism (e.g., you should save two drowning children rather than one ceteris paribus) and we would like some account of how that consequentialism is supposed to work without bringing cluelessness back in."