The pedantic mathematician in me though didn't like the concept in the "Intermediate Value Theorem" section being described with that name. I'm not sure that is actually the theorem being relied on here.
A couple of reasons:
IVT only applies to functions from the real numbers to the real numbers, whereas the example you're using it on is a mapping from a discrete set to a discrete set (or at a stretch, from rationals to rationals). IVT doesn't apply there (e.g. the function x -> x^2 on the rationals between x=0 and x=2 goes from 0 to 4 without ever passing through 2).
I don't think the IVT is "trivially easy". It's a theorem that the formal definition of continuity aligns with our intuition (which is not trivial), and really it is only true because the real numbers have been specifically constructed to make it true (it is not true over the rational numbers), and the construction of the real numbers is very weird and non-trivial!
The actual concept you're relying on in that section is actually a lot simpler I think. It's essentially just the "paradox of the heap". As you say, if the value of N things is V, and the value of N+M things is W, then the sum of the changes of each addition to N must add to (W-V). You don't need calculus for that. It's not the Intermediate Value Theorem. It really is just "trivially easy" I think. I'm not sure if that theorem has a name.
Ironically I might also be guilty of using some technical terminology incorrectly here!
I had in mind the discussion on valuing actions with imperceptible effects from the "Five Mistakes in Moral Mathematics" chapter in Reasons+Persons (relevant to all the examples mentioned in the IVT section of this post), where if I remember right Parfit makes an explicit comparison with the "paradox of the heap" (I think this is where I first came across the term).
It feels the same in that for both cases we have a function from natural numbers (number of grains of sand in our potential heap, or number of people voting/buying meat) to some other set (boolean 'heap' vs 'not heap', or winner of election, or number of animals harmed). And the point is that mathematically, this function must at some point change with the addition of a single +1 to the input, or it can never change at all. Moreover, the sum of the expected value of lots of potential additions must equal the expected value of all of them being applied together, so that if the collective has a large effect, the individual effects can't be smaller, on average, than the collective effect divided by the number of consituents.
I suppose the point is that this paradox is non-trivial and possibly unsolved when the output is fuzzy (like whether some grains of sand are a heap or not) but trivially true when the output is precise or quantitative (like who wins an election or how many animals are harmed)?
The relatively more orthodox view amongst philosophers about the heap case is roughly there is a kind of ambiguous region of successive ns where it is neither true nor false that n grains make a heap. This is a very, very technical literature though, so possibly that characterization isn't quite right. None of the solutions are exactly great though, and some experts do think there is an exact "n" where some grains become a heap.
Also, pretty much every empirical outcome is potentially fuzzy in some possible situation. (Elections can be tied, whether an experience is painful and hence harm or neutral can be unclear etc.)
Maybe, although an election being tied is about the only way that particular example can be fuzzy, and there is a well defined process for what happens in that situation (like flipping a coin). There is ultimately only one winner, and it is possible for a single vote to make the difference.
Whether an experience is painful or not is extremely unclear, but if your metric is just something like "number of animals killed for meat each year" then again that is something well defined and precise, and it must in principle be possible to change it with an individual purchase.
It can also be indeterminate over a short time who the winner of an election is because the deciding vote is being cast and plausibly there is at least some very short duration of time where it is indeterminate whether the process of that vote being cast is finished yet. It can be indeterminate how many animals were killed for food if one animal was killed for multiple reasons of which "to eat" was one reason but not the major one. Etc. etc.
Thanks! I agree the math isn't exactly right. The point about x^2 on the rationals is especially sharp.
The problem with calling it "the paradox of the heap" is to make it sound like an actual paradox, instead of a trivially easy connection re:tipping points. I wish I had a better terminology/phrase for the connection I want to make.
A term that means something similar to theory of mind but something dissimilar to how you are using the term here (and how the term is most often used):
Theory of mind, mentalization, cognitive empathy, and perspective taking are, of course, not actually "rare" but are what almost all people are doing almost all the time. The interesting question is what kinds of failures you think are common. The more opinionated you are about this, and the more you diverge from consensus opinions of experts such as psychologists and researchers in social work, the more likely you are to be wrong.
Whether people are correctly mentalizing or perspective taking or engaging in accurate cognitive empathy is often a controversial and contested question. These disagreements can't be resolved simply by invoking the concept of theory of mind (or a similar term or concept).
For example, is misogyny or sexism a form of hatred? And if a person or group is taken to have misogynist or sexist views, is it accurate to say that person or group hates women? Are people who make such claims mentalizing incorrectly by misdiagnosing misogyny as hatred of women, or are you mentalizing incorrectly by misdiagnosing their diagnosis as incorrect mentalization? I don't think disputes like this can be resolved by just making uncontroversial assertions about what theory of mind is. And if you're using contested examples like this as the paradigmatic examples upon which the rest of your exploration is built, then your treatment of the topic is probably going to end up assuming its conclusions — and failing to persuade anybody who didn't already accept those conclusions from the outset.
I'm not sure the concept of net present value meaningfully tells us anything new about world hunger or global poverty. The connection between the concepts of net present value and world hunger is very loose. It is true that there are lots of people (including random pseudonymous people on Twitter) who don't understand important concepts in accounting, finance, and economics. Failure to understand these concepts may lead to bad analysis. But development economists obviously understand these concepts, so if the point is to understand world hunger or global poverty, it would be a better idea to just read an introductory text on international development than to think further about how the concept of net present value might or might not shed new light on global poverty.
I personally don't find any value in Grice's maxims. There is a danger in being too general, too abstract, and too vague in the advice you give, such as that it comes close to boiling down to 'do good things and don't do bad things'. Or in saying things that are so obvious, such as 'say true things and don't say untrue things', that the advice is pointless, since everybody already knows that.
I find the slogan "ideas matter" to be unremarkable in a similar way as 'say true things and don't say untrue things'. I don't think anybody disagrees that ideas matter. I would say everyone agrees with that.
If someone were presenting to me the thesis "ideas matter" and it were a somewhat novel or interesting thesis, I would expect it to be something along the lines of looking at ideas in history that had a surprisingly large impact. For comparison, I recently watched a fascinating interview about the historical importance of textiles. I was surprised by so many things in that interview, I learned a lot. That video definitely made me think textiles matter a lot more than I realized. It supported the thesis "ideas in textile innovation matter". What would a case for the thesis "ideas matter" look like? Maybe something like that, but more general. However, I think it's so intuitive and widely believed that science, technology, politics, religion, and scholarship are important, it would be hard to present a case that is surprising or novel enough to make most people think that "ideas matter" non-trivially more than they already did.
Overall theme of this comment:
It's hard to innovate beyond the state of the art, and it's easy to overstate how novel one's own insights are, or to overstate how well-supported one's controversial opinions are.
That isn't a reason not to explore or celebrate interesting ideas, of course. But it is a reason to change certain aspects of the presentation, such as acknowledging that theory of mind is ubiquitous, not "rare", and acknowledging that your own personal ideas about what failures of theory of mind are common might be either completely non-novel or wrong (or, if not wrong, at least highly controversial and beyond just an exposition of the concept of theory of mind).
I'm not trying to dampen your enthusiasm, but trying to forestall some combination of a) presenting old hat as novel or revelatory in a way that verges on plagiarism and b) presenting controversial and unsupported (or minimally supported) ideas, including some ideas original to you, as being as well-supported as the old hat. I'm not sure either (a) or (b) is where you were going with this post, but I sort of got that feeling from it. To be a science communicator (or economics communicator, etc.) and to be a theorist are different roles, and we don't want to get them mixed up such that our communication of old, established ideas is mistaken for original theory or that our original theory, which is not yet supported and may be false, is mistaken for old, established ideas.
I think your comment largely addresses a version of the post that doesn't exist.
In brief:
I don't think I claimed novelty; the post is explicitly about existing concepts that seem obvious once you have them. I even used specific commonly known terms for them.
Theory of mind, mentalization, cognitive empathy, and perspective taking are, of course, not actually "rare" but are what almost all people are doing almost all the time. The interesting question is what kinds of failures you think are common. The more opinionated you are about this, and the more you diverge from consensus opinions of experts such as psychologists and researchers in social work, the more likely you are to be wrong.
The post gave specific examples of people with the capacity for ToM nonetheless failing to consistently apply it to political outgroups, foreign adversaries, story characters etc. Also the specific wording I wrote was:
The core idea is very simple: treat other agents as real. It sounds banal, until you realize how rare it can be, and how frequently people mess up."
You harp on the word "rare" but miss the surrounding context. You consistently make technically true but irrelevant points.
so if the point is to understand world hunger or global poverty, it would be a better idea to just read an introductory text on international development than to think further about how the concept of net present value might or might not shed new light on global poverty.
Are you seriously implying that it takes less effort to read an entire textbook on developmental economics than it is to write a paragraph on a related question? Besides, that wasn't the point of the post anyway, which was more like "here's a specific conceptual error people make, NPV dissolves it."
I don't think anybody disagrees that ideas matter. I would say everyone agrees with that.
This blog post initially grew out of a conversation with a popular blogger about whether ideas actually matter. It's also commonly believed in Silicon Valley that ideas are almost irrelevant compared to execution.
I personally don't find any value in Grice's maxims.
I would like to respectfully request that you not engage with me in the future due to your violation of civility norms, and I'll likewise not engage with you in the future. Take care.
This is the third in a sequence of posts taken from my recent report: Why Did Environmentalism Become Partisan?
Summary
Rising partisanship did not make environmentalism more popular or politically effective. Instead, it saw flat or falling overall public opinion, fewer major legislative achievements, and fluctuating executive actions.
Public Opinion...
I think right now EAs might be making a significant mistake by paying insufficient attention to the political realm. As EAs we tend to figure out what’s most impactful for us to work on and focus hard. That’s great! But there are various actions that are ‘non-delegatable’ - the extent to which an individual can do the action is limited (like voting, going to a protest, making hard money contributions to particular campaigns). It might be useful if we were all more in the habit of doing variou...
This post presents the executive summary from Giving What We Can’s impact evaluation for 2025. At the end of this post we share links to more information, including the full report and...
Merry Christmas and Happy Holidays to those who celebrate, and a Fun Friday to those who don’t! Today I want to talk about "unknown knowns" : "obvious" ideas that are actually hard to discuss because they're invisible when you don't have them, and then almost impossible to unsee when you do.
There are a number of implicit concepts I have in my head that seem so obvious that I don’t even bother verbalizing them. At least, until it’s brought to my attention other people don’t share these concepts.
It didn’t feel like a big revelation at the time I learned the concept, just a formalization of something that’s extremely obvious. And yet other people don’t have those intuitions, so perhaps this is pretty non-obvious in reality.
Here’s a short, non-exhaustive list:
Intermediate Value Theorem
Net Present Value
Differentiable functions are locally linear
Grice’s maxims
Theory of Mind
If you have not heard any of these ideas before, I highly recommend you read up on the relevant sections below! Most *likely*, they will seem obvious to you. You might already know those concepts by a different name, or they’re already integrated enough into your worldview without a definitive name.
However, many people appear to lack some of these concepts, and it’s possible you’re one of them.
As a test: for every idea in the above list, can you think of a nontrivial real example of a dispute where one or both parties in an intellectual disagreement likely failed to model this concept? If not, you might be missing something about each idea!
Concept: If a continuous function goes from value A to value B, it must pass through every value in between. In other words, tipping points must necessarily exist.
This seems almost trivially easy, and yet people get tripped up often:
Example 1: Sometimes people say “deciding to eat meat or not won’t affect how many animals die from factory farming, since grocery stores buy meat in bulk.”
Example 3: Sometimes people say that a single vote can’t ever affect the outcome of an election, because “there will be recounts.” I think stuff like that (and near variants) aren’t really things people can say if they fully understand IVT on an intuitive level.
The core mistake? People understand there’s some margin where you’re in one state (eg, grocery store buys 2000 pounds of chicken) and some margin where you’re in another state (eg, grocery store buys 3000 pounds of chicken). But without the IVT, people don’t realize there must be a specific decision someone makes that tips the situation from the first state to the second state.
Note that this mistake (IVT-blindness) is recursive. For example, sometimes people understand the reasoning for why individual decisions might matter for grocery store orders but then don’t generalize, and say that large factory farms don’t make decisions on how many animals to farm based on orders from a single grocery store.
Interestingly, even famous intellectuals make the mistake around IVT. I’ve heard variants of all three claims above said by public intellectuals.[1]
Net Present Value
Concept: The value today of a stream of future payments, discounted by how far away they are. Concretely, money far enough in the future shrinks to nearly nothing in present value, so even infinite streams have finite present value[2].
Example 1: Sometimes people are just completely lost about how to value a one-time gain vs benefits that accumulate or compound over time. They think the problem is conceptually impossible (“you can’t compare a stock against a flow”).
Example 2: Sometimes people say it’s impossible to fix a perpetual problem (e.g. SF homelessness, or world hunger) with a one-time lump sum donation. This is wrong: it might be difficult in practice, but it’s clearly not impossible.
Example 3: Sometimes people say that a perpetual payout stream will be much more expensive than a one-time buyout. But with realistic interest rates, the difference is only like 10-40x.
Note that in many of those cases there are better solutions than the “steady flow over time” solution. For example, it’d be cheaper to solve world hunger via agricultural and logistical technology improvements than the net present value of “feeding poor people forever.” But the possibility of the latter creates an upper bound for how expensive this can be if people are acting mostly rationally, and that upper bound happens to be way cheaper than current global GDP or wealth levels.
Differentiable functions are locally linear
Concept: Zoom in far enough on any smooth curve and it looks like a straight line.
Example 1: People might think “being risk averse” justifies buying warranties on small goods (negative expected value, but shields you from downside risks of breaking your phone or something). But this is not plausible for almost any realistic risk-averse utility function, which becomes clear once you realize that any differentiable utility function is locally linear.
Example 2: People often have the intuition that altruists should be more careful with their money and more risk-sensitive than selfish people, even though the opposite is true. Altruistic people care about global welfare, which is a large function, so zoomed in, almost any individual altruist’s donation budget is linearly good for the world at large.
Example 3: People worry about “being pushed into a higher bracket” as if earning one more dollar could make them worse off overall. But tax liability is a continuous (piecewise linear) function of income. No additional dollar in income can result in greater than one dollar of tax liability, other than very narrow pathological cases.
Understanding that differentiable utility functions are locally linear unifies a lot of considerations that might otherwise confuse people, for example, why one sometimes ought to buy insurance for health and life but almost never for small consumer products, why altruistic people should be more risk-seeking with their investments, why bankroll management is important for poker players, etc.
Grice’s maxims
Concepts: Grice actually has four maxims:
Quantity (informativity): Say enough, but not more than needed.
Quality (truth): Only say what you believe to be true and can be supported.
Relation (relevance): Be relevant.
Manner (clarity): Be clear, brief, and orderly.
I think disputes where one or both sides don’t follow each of Grice’s maxims should be fairly self-explanatory.
Many forms of trolling break one or more of these maxims, but not all of them. For example, a gish gallop is breaking the maxim of informativity. Bringing up Hilary Clinton’s emails, or the last Trump escapade, in an otherwise non-political discussion is breaking the maxim of relevance. The bad forms of continental philosophy often break the maxim of manner, which is why many analogize their writings to trolling. And of course, many trolls lie, breaking the maxim of quality.
For a longer, and somewhat ironic, meditation on the importance of Grice’s maxims, consider reading my earlier post:
Concept: ToM has many components, but the single most important idea is that other people are agents too. Everybody else has their own goals, their own model of how the world works, and their own constraints on what they can do.
Example 1: Sometimespeople ascribe frankly implausible motivations to their enemies, like “Republicans just hate women”, “Gazans don’t care about their children,” “X group just wants to murder babies” etc.
Example 2: Sometimes people don’t even consider that their enemies (and allies, and neutral third parties) even have motivations at all. The Naval War College Historian Sarah Paine calls this “half-court tennis”: sometimes US government officials and generals think about war and peace in relation solely to US strategic objectives. They don’t even consider that other countries have their own political aims, and do not primarily define their own politics in relation to US objectives.
Example 3: Do you often feel like characters in a novel seem “flat?” Like they’re characters who think they should be characters in a novel to advance a narrative point, not fully-fleshed out people with their hopes and dreams.
The core idea is very simple: treat other agents as real. It sounds banal, until you realize how rare it can be, and how frequently people mess up.
I think a full treatise on a theory of mind failures and strengths is worthy of its own blog post, and that’s what I’m working on next! Subscribe if you’re interested! :)
Why this all matters
Well, first of all, I think all of the concepts above are important, and neat, and it’d be good if more of my readers know about them!
More importantly, I think ideas matter. I deeply believe that ideas are extremely important and behind much of civilizational progress (and backsliding).
This is one of the central themes of this blog: ideas matter, and if we try harder and work smarter, if we approach every problem with simultaneous dedication and curiosity, together we can learn more ideas, integrate them into our worldviews, and use those ideas to improve our lives, and the world.
I don’t just mean big, all-encompassing, ideological frameworks, like Enlightenment or Communism. I also don’t just mean huge scientific revolutions, like evolution or relativity.
I mean small ideas, simple concepts like the ones above, that help us think better thoughts and live better lives.
I’m interested in a category I think of as Unknown Knowns: concepts that, once acquired, feel less like models you learned and more like obvious features of reality. They’re invisible until you have them, and then, once acquired, almost impossible to unsee. So you never truly notice them.
Today, almost 2000 years after some Jewish dude was nailed to a tree for championing the idea of how great it would be to be nice to people for a change, I want to actually see these ideas again. I want to take some time to appreciate all the ideas that have made my reality better, and all the people who made sacrifices, great and small, to find and propagate those ideas.
People interested in this might be interested in @Peter McIntyre's project which lists 52 useful concepts. https://mcntyr.com/blog/52-concepts-cognitive-toolkit