I was about to make a post to the forum when I came across the Review of ITN Critiques, and I soon after thought of a counter that I believe accounts for many (or all?) of the various threads in that article. I have not had time to take the arguments further, and I have other drafts I want to work on since this is the last day of draft amnesty week. Thus, I just decided to post it in this form for now. 

I think a large portion of the critiques are aimed at the more intuitive, less formal versions of the ITN framework, but I respond to them as if they are critiqing the formal version anyway.  So without reading all the posts that the review mentions in detail, here is my take. 

 

A Formal Equivalent of the ITN Framework

First, I want to make an analysis basreed on an observation. Let's take a look at the numerical version of the ITN Framework advanced by 80,000 hours:

I prefer to work with more compact notation, so I define the terms such that:

Then, if we cancel terms we get that:

Then we see that:

where d is an infinitesimal change. 

Similar to before, we still have that:

For those that don't already know derivatives from calculus, we have that $dG/d\$$ really just means a small(infinitesimal) difference in good done per small difference in the amount of money donated. The top line is equivalent to the chain rule.  

Consequences

Why does this formulation matter? 

Because it helps one realize that any of the individual terms in the equation is given by some functional formula. If we take neglectedness to be $\frac{dR}{d\$}$, then we see that this is really just a functional derivative dependent on the shape of the original formula. Theoretically speaking, the function can take on any shape we want want it to. If, for example, we only want our function to take on the shape of diminishing returns "beyond some threshold specific to a given cause" due to the fact that there are setup costs, we can, as is the case with figure 1:
 

This is exactly why it is not the case that Neglectedness improperly assumes diminishing marginal returns, at least for the formal version of the framework. Neglectedness in the formal version of the framework is a bit more like the concept of leverage, as will macAskill points out. 

I do think that we should predict, in expectation, what this curve is going to look like, rather than focusing only on the absolute marginal returns in any given scenario like in the curve above. That is, we want the area under the curve for neglectedness. Focusing on the absolute marginal returns in any given scenario is characteristic of a greedy algorithm. 

The reality is that in practice we rarely know the exact curve as in figure 1, and so the greedy algorithm is most often the best we have, combined with one's own intution.

It is additionally important to note that since we can never know the true Importance, neglectedness, and tractability, we are really after the expected value of these terms. A key insight is that these terms are not independent:

That is to say, these terms are not necessarily independent of each other^[1]. With that said, I think the effect of this dependence is overblown a bit, 

"Neglectedness improperly assumes diminishing marginal returns"

Neglectedness prioritizes change over magnitude of marginal returns

Here they say:

One cause may face diminishing marginal returns (MR), yet have high absolute MR. Another might have low absolute MR but enjoy increasing MR, perhaps due to high setup costs. Neglectedness favors the latter cause over the former, which may not be appropriate [2] [3]. 

But I think this mistakes what neglectedness is actually measuring. 

1. ^{^}

   A closer approximation would be given by:

   

   But this only accounts for linear correlations. A more accurate characterization would be:

   

   However, this last formulation is likely to be extremely hard to calculate, and so at most a few of the terms from the covariance equation could be included. Both formulations are extremely unwieldy, and likely impracticle to use.