Thanks for posting! I especially like the part you mentioned on how it's possible to slip into a mindset of justification/motivated reasoning of past actions as being cost effective for happiness for productivity! Daniel Kirmani made a comment here more in depth about it that makes me think about the book The Elephant In The Brain that expands on this idea in depth(you may have already heard of it).
Thanks for posting this! I really appreciate it.
I want to highlight some relevant posts:
I think they're especially relevant for this section:
But possible self-defeating dynamics aren’t the only issue. Another is that pressure to justify everything can cause people to come up with justifications after the fact. What do you do if you only heard about EA six months ago, and already made some decisions beforehand? What do you do if you just didn’t think very hard about working at a random tech company rather than something “more EA”? What if you just went to the party because you wanted to?
Justifying, in these cases, is also a way to get practice... in motivated reasoning.
Linking to Spencer Greenberg's excellent short talk on intrinsic values:
Spencer claims, among other things, that
I like this post. Another post I'd recommend on related topics is Act utilitarianism: criterion of rightness vs. decision procedure. Unfortunately it lacks a summary (though it is at least quite short), but here's one excerpt:
As many utilitarians have pointed out, the act utilitarian claim that you should ‘act such that you maximize the aggregate wellbeing’ is best thought of as a criterion of rightness and not as a decision procedure. In fact, trying to use this criterion as a decision procedure will often fail to maximize the aggregate wellbeing. In such cases, utilitarianism will actually say that agents are forbidden to use the utilitarian criterion when they make decisions.
I also like and recommend Purchase Fuzzies and Utilons Separately.
For more on criterion of rightness vs decision procedure
LOVE this post. The Julia Wise one you link to is my fav. And now this is an excellent pairing for that. I will be sure to share when I see people doing what you describe, or worrying about it, which will be sooner than later !!!
Thank ya Thomas!!
Thanks to Miranda Zhang and Teddy Tawil for feedback on this post.
EA shares a key similarity with large family gatherings: you constantly feel the need to justify your life choices. You might have a really good justification, but you might also feel like you need to make something up.
I think it’s good for people to give explanations of their choices. After all, EA is about making the choices which will maximize your impact, and many of these choices are career-related. Your career is a major part of your life, and if you say you’re choosing a career path on the basis of EA principles, people in EA will sometimes ask you to give an explanation for why you're choosing that particular path. This isn't usually because they are trying to expose you, but because they genuinely want to hear your perspective and help you have the most impact that you can.
In fact, some people fill out applications just to speak with someone who can help you make life choices that are more justifiable (this is called 80,000 Hours advising).
A justification is supposed to indicate how you’ve thought about what you’re doing and why you’re doing it instead of any other options you had. In EA, this usually means why you think it’s likely to have the highest impact. Thinking carefully about your life choices is one way to be able to produce a legible justification, and thinking carefully is certainly an extremely important aspect of having impact. One way to practice thinking is to practice explaining.
But I’ve noticed that sometimes, people in EA try to justify seemingly everything.
Why are you ordering food instead of cooking? Because it saves me time so I can spend more time working. Why do you spend so much time gardening? Because it keeps me sane so I can do more one-on-ones with new group members. Why are you going to that party tonight? Because it will make me happier, which will help me produce more long-run undiscounted utility.
I think it can be very worthwhile to think about the major parts of your life on a semi-regular basis. Are they valuable to you? Do they help you achieve your goals (Julia Wise provides an excellent justification for having more than one)? Are they worth changing? If you think hard about decisions as large as your career, you will do much more good in expectation. The unexamined life is not worth many impact points.
However, even if you are an ardent utilitarian, it doesn’t make sense to make every single decision on the basis of a calculated utility for that decision. Many people have written great things about this, so I will not try to justify it fully here. But the basic gist is that act utilitarianism can be self-defeating: it can be time-consuming and emotionally difficult to decide every decision on the basis of the greater good, and that’s not even considering the risk of the unilateralist’s curse. Because of that, being a strict act utilitarian can reduce the utility of your acts.
But possible self-defeating dynamics aren’t the only issue. Another is that pressure to justify everything can cause people to come up with justifications after the fact. What do you do if you only heard about EA six months ago, and already made some decisions beforehand? What do you do if you just didn’t think very hard about working at a random tech company rather than something “more EA”? What if you just went to the party because you wanted to?
Justifying, in these cases, is also a way to get practice... in motivated reasoning. Why did you go to two parties last weekend? Maybe you just need two weekly parties to be happy enough to work. Why were you spending so much time trying to get an A in differential equations that you forgot to apply to that internship? Well, a top 5 PhD program requires a high GPA. These justifications could be true, but are they really why you did the thing? If they aren’t, making justifications can get you into the habit of making up fake reasons for everything you do, reducing your ability to think about which reasons aren’t fake. Your poor justifications might also erode people’s ability to believe you when you really have thought about something really hard.
My proposal: don’t feel the need to justify everything through the lens of impact. If you don’t have a good justification, just say so. Of course, this goes both ways. If somebody else doesn’t want to provide a justification for something, respect that. It’s better for people to be honest than to make up an answer after the fact, and for minor things, it might even be better to not think too hard about it at all.
So the next time you feel the need to justify something you did, and you don’t have a good explanation, consider giving the same answer a moody teenager would give at the dinner table: "I did it because I felt like it."
Reminder that split-brain experiments indicate that the part of the brain that makes decisions is not the part of the brain that explains decisions. The evolutionary purpose of the brain's explaining-module is to generate plausible-sounding rationalizations for the brain's decision-modules' actions. These explanations also have to adhere to the social norms of the tribe, in order to avoid being shunned and starving.
Humans are literally built to generate prosocial-sounding rationalizations for their behavior. They rationalize things to themselves even when they are not being interrogated, possibly because it's best to pre-compute and cache rationalizations that one is likely to need later. It has been postulated that this is the reason that people have internal monologues, or indeed, the reason that humans evolved big brains in the first place.
We were built to do motivated reasoning, so it's not a bad habit that you can simply drop after reading the right blog post. Instead, it's a fundamental flaw in our thought-processes, and must always be consciously corrected. Anytime you say "I did X because Y" without thinking about it, you are likely dead wrong.
The only way to figure out why you did anything is through empirical investigation of your past behavior (revealed preferences). This is not easy, it risks exposing your less-virtuous motivations, and almost nobody does it, so you will seem weird and untrustworthy if you always respond to "Why did you do X?" with "I don't know, let me think". People will instinctively want to trust and befriend the guy who always has a prosocial rationalization on the tip of his tongue. Honesty is hard.
Yes, people will always have motivated reasoning, for essentially every explanation of their actions they give. That being said, I expect it to be weaker for the small set of things people actually think about deeply, rather than things they're asked to explain after the fact that they didn't think about at all. Though I could be wrong about this expectation.
If you spend a lot of time in deep thought trying to reconcile "I did X, and I want to do Y" with the implicit assumption "I am a virtuous and pure-hearted person", then you're going to end up getting way better at generating prosocial excuses via motivated reasoning.
If, instead, you're willing to consider less-virtuous hypotheses, you might get a better model of your own actions. Such a hypothesis would be "I did X in order to impress my friends, and I chose career path Y in order to make my internal model of my parents proud".
Realizing such uncomfortable truths bruises the ego, but can also bear fruit. For example: If a lot of EAs' real reason for working on what they do is to impress others, then this fact can be leveraged to generate more utility. A leaderboard on the forum, ranking users by (some EA organization's estimate of) their personal impact could give rise to a whole bunch of QALYs.
This is a good point which I don't think I considered enough. This post describes this somewhat.
I do think the signal for which actions are best to take has to come from somewhere. You seem to be suggesting the signal can't come from the decisionmaker at all since people make decisions before thinking about them. I think that's possible, but I still think there's at least some component of people thinking clearly about their decision, even if what they're actually doing is trying to emulate what those around them would think.
We do want to generate actual signal for what is best, and maybe we can do this somewhat by seriously thinking about things, even if there is certainly a component of motivated reasoning no matter what.
If this estimate is based on social evaluations, won't the people making those evaluations have the same problem with motivated reasoning? It's not clear this is a better source of signal for which actions are best for individuals.
If signal can never truly come from subjective evaluation, it seems like it wouldn't be solved by moving to social evaluation. One thing that would seem difficult would be concrete, measurable metrics, but this seems way harder in some fields than others.
(Intersubjective evaluation - the combination of multiple people's subjective evaluations - could plausibly be better than one person's subjective evaluation, especially if of themselves, assuming 'errors' are somewhat uncorrelated.)