Yep, and I think I disagree that you're justified in doing that, but I'd be interested in reading the case for it. 

I think we'll end up going in circles - my point is that cluelessness is a problem with the reasoning that gets you to your preferred action, so if it is true, then the reasoning is faulty and the preference is unjustified. 

Also, since you can still have preferences if you believe in cluelessness, I don't quite know how you'd compare between your preference to act deontologically in cluelessness world and your preference to maximise impartial welfare in non-clueless world. Both would be good preferences in their respective worlds. IDK how the meta-normative comparison should go here. 

Edit: to clarify, I think our crux might be at this metanormative level, i.e. at comparing cluelessness world preferences with non-cluelessness world preferences. I realise I'm pretty agnostic on how to do that, but a good argument here could cause me to find the wager response to cluelessness more convincing. 

Are you assuming that the previously preferred action would still have some normative force behind it? I think that's the bit that's confusing me. The cluelessness argument (if it goes through - idk if it does or not, hence competition) attacks your reasons for preferring that action, so you're back to the drawing board. It doesn't make much sense to still insist on the action when you find out that your reasons for preferring it are flawed. 

But this argument is attacking the exact reasons you thought it was good to do the thing in the first place no? I.e. that you expected it would have positive consequences?

I think you're basically describing the wager argument, which I think is in this post approximated by the 'heuristics' counterargument. 

I'd be keen to see someone develop the wager argument more. (i.e. 'wager' like in Pascal's wager)

The short answer is that if cluelessness is true you aren't justified in doing what you'd do if it wasn't true. You can still do it, but you aren't being responsive to the right reasons. 

Not quite - the idea is that we run into the cluelessness objection when we act for impartial altruistic reasons. Deontological reasoning, and virtue ethics reasoning are different in that when we reason that way, we aren't making claims about consequences. 

If you want to care about the long-run impartial consequences of your actions, then you need to answer Anthony's argument, either by rejecting it for a specific reason, or finding a way to reason about impartially altruistic actions without running into the problems he outlines. 

PS- Your objection makes a lot of sense, but it came across a bit ruder/ more dismissive than necessary. I'm guessing that's why it's been downvoted, and it's a bit of a shame since I'm sure other readers have similar reactions. 

I'm curating this because I think it's a valuable and sober reflection on the (possible, future) funding situation. It's (always) hard to reason about a big event that might not happen, or might not happen the way you think, and I appreciate Sam's more bayesian approach here. 

Some other great posts on the (possible, future) funding situation, if you'd like to read more:

• Reflections on Anthropic and EA
• Incoming money, integrity, and collective action problems

"I'm not sure why we should expect the group to be well-calibrated when no individual is?" - Something something marketplace of ideas? An analogy is a court, where the prosecution and the defence both have their conclusions assigned to them beforehand. They are both epistemically vicious, in opposite ways. Then the idea is that the best arguments win on their merits. I'm not sure quite how this analogy would fit though (I've had false starts writing a blog post on this a few times). 

"I certainly don't endorse doing nothing" Yep I get that this argument is only one consideration, but my point extends to "trying less hard" as well I think. 

After our chat the other day, it'd be great to get your takes on this @NickLaing! Especially - could nurses + AI leapfrog GPs? Seems like a clear counter-example to this case, if it works. 

Interesting post!

I suspect that the epistemic health that matters is on the level of the group and not the individual. I.e. I care about the net effect of the group's actions, and I don't much care whether individuals are rational, as long as the group acts rationally. With that framing, would it be better if everyone individually 'tried hard' or not? 

Seems like it'd net out to being better if the correct course of action produces much more utility than the aggregate disutility of the rest of the actions we are likely to take. However, if it's very easy to do badly wrong, doing less might be the better strategy. 

One other point though: When you take negative responsibility seriously (as we should), you dispense with the idea of a neutral option, or the choice to do nothing. Bracketing the epistemics-distorting point, there isn't necessarily a difference in the expected (sign-neutral) impact of doing 'nothing' and trying hard.