Could you list out those statements? Are you talking about the article, or my comments? I would certainly not want to intentionally mislead, nor be outright incorrect.
Sure:
Cash balance methodology
You say:
Dividing $26,260 by GiveWell’s end of year cash balance in 2019—the $41,101,155 we calculated in the previous section—produces an upper bound interest rate of 0.0006389, or 0.064%.
Multiplying GiveWell’s $41,101,155 in cash at the end of 2018 by a 2.16% rate of interest yields a total of $887,785 in potential interest earnings which is nearly 34 times higher than the actual investment income GiveWell earned.
Subtracting the $26,260 upper bound on how much GiveWell actually earned on its cash in 2019 results in an annualized opportunity cost of $861,525.
Incorrect and misleading statements in these four paragraphs:
First paragraph: Dividing interest earned over the year by end of year cash is not an upper bound estimate interest. It's actually more like a lower bound estimate, because it implicitly assumes that the cash level at end of year was the same as the average cash level throughout the year.
Since GiveWell is a grant-making organization that's a very dangerous assumption to make (and their 990 form shows them making ~$30m in grants over the course of the year).
You acknowledge this in the next section, but then propose an equally questionable workaround:
GiveWell earns interest on its average cash balance during the year, rather than its cash balance at one point in time like the end of the year. We can better approximate this figure by averaging GiveWell’s beginning of year and end of year zero-interest and non-zero-interest cash and then summing them. This produces an estimated average cash balance in 2019 of $33,441,682.
In your reply to me you suggest:
You may not be understanding what the article means by "cash balance." As I make very clear in the article, we are using the organization's estimated average daily cash balance to calculate the historical and future opportunity costs. The average daily cash balance is the amount that an organization literally makes interest on; I can't think of a better number to use [emphasis added]to calculate the interest an organization previously made and could be making in the future.
Even this approach seems incorrect or misleading:
The page for their Maximum Impact Fund shows the grants they make (which would come out of the cash balance shown in the 990), and it seems to show a couple of months lag between donations and grants.
The same page shows that funds received in the final three months of 2019 were substantially greater than those received in the first nine months of 2019, which is another reason that the end-of-2019 cash balance paints a misleading picture.
Second paragraph: The year you've chosen for your analysis happens to be the year with the highest cash rates in the past 12 years. This isn't acknowledged, and you don't backtest your strategy in other years (e.g. all the years where rates were ~0%).
Third & fourth paragraphs: Same analytical error as first paragraph.
Potential returns of other investment options
You say:
The 2.16% U.S. federal funds rate in 2019 is one of the most conservative interest rates possible. Higher interest rates would result in significantly higher annual opportunity costs. Our top recommendation for an ultra-short-term bond ETF (JPST) in our recommendation article published in early 2019 earned 3.34% in 2019, according to Portfolio Visualizer. There were no drops in account value on a monthly basis (0% maximum drawdown). Multiplying GiveWell’s $30,033,677 reasonable savings balance by 3.34% generates an opportunity cost of over one million dollars per year, at $1,003,124.
Citing the one year, backward looking return of JPST is misleading, and citing the one year drawdown performance is naive:
There's nothing about that asset that makes its one year backward-looking return able to be extrapolated. Indeed this is basically the equivalent of picking a different asset that you know now performed well over a one year period and using that to estimate opportunity cost. Why not Bitcoin in 2017 or Tesla stock in 2020?
And re citing a year of drawdown data - the fund had a drawdown in March 2020! If you look at a similar index with a longer timeseries you'll see there have been drawdowns, even against the backdrop of a period of secularly falling interest rates that juiced bond returns.
What's more, that particular fund tracks a bond benchmark but is actively managed, meaning its managers can take more risk than the benchmark implies (and charge a management fee for doing so).
Using an even higher interest rate of 5.51%, which is what a very conservative portfolio of 20% stocks and 80% bonds earned on average (CAGR) from January 2000 to December 2019 calculated using Portfolio Visualizer, the one-year opportunity cost on GiveWell’s 2019 reasonable savings balance would be $1.65 million dollars a year. The 5.51% figure is an approximation of GiveWell’s expected opportunity cost rather than what it would have experienced, which would have been a small loss of -1.15% in 2018 followed by a 13.01% gain in 2019. For those curious about how this portfolio would have fared during COVID-19, the maximum negative month-over-month change in account value was -3.20% since the beginning of 2020; the portfolio is now up 9.11% for the YTD period ending November 30, 2020.
This analysis suffers from the same problem. At least there's an attempt to look at a window longer than one year, but again it's a cherry-picked period of 20 years, and it's backward looking returns data implied to be forward-looking. You also mention the drawdown in 2018. What was the maximum drawdown in 2008 to 2009?
Given the organization is granting funds a couple of months after receiving them, it has little ability to tolerate drawdowns, making comparisons to a higher risk fund incorrect to misleading.
Conclusion and staff time
You say:
Regardless of the estimation method, it’s very likely the counterfactual impact of changing GiveWell’s financial practices is in the millions of dollars. Not bad for a time investment that should take between 1–10 hours of staff time to set up, and 0–2 hours of staff time to administer on an annual basis!
As shown above, getting a counterfactual impact number in the millions of dollars is not "very likely... regardless of the estimation method". That statement is somewhere between incorrect and misleading. For instance if the cash balance is 10% of what you cite, and the interest rate is more like 0.50%, it doesn't hold up.
The staff time estimate is also somewhere between incorrect and misleading:
If it's just opening bank accounts, then the impact is GiveWell on average earning roughly the prevailing bank interest rate (close to zero for much of the relevant period)
If instead you want them (without the benefit of hindsight that you employ) picking actively managed JP Morgan bond funds, or creating and balancing a bond + stock fund, then I'd hazard it's more than two hours of work per year!
So either it's not much effort, and not much return, or there can be higher returns, but clearly it's more than 2 hours a year of effort.
This article makes several statements that I interpret as somewhere between intentionally misleading and outright incorrect.
For instance, this reply says the 2.19% Fed Funds rate was the prevailing rate in 2019, and isn't intended to be a forecast of the future. But since we know the Fed Funds rate is 0% today, why would that rate be used to argue for a forward-looking change in behavior?
Similarly, the 2019 return of a short-term bond fund gives literally no information about what returns a similar fund would offer today. Indeed the beauty of the bond market is that it does offer guaranteed future returns if you hold a bond to maturity. As of today, US government bonds (which carry no risk of default) offer a guaranteed annualized return of:
0.08% if held for 3 months,
0.10% if held for 12 months,
0.13% if held for 2 years,
0.94% if held for 10 years
So assuming an organization needed to make a grant within 12 moths, they could currently expect to make 0.10% on their cash balance. Needless to say this is 1/10th of 1%, or <1/20th of the figure cited in the post. And it seems a stretch to assume organizations hold cash for 12 months before granting it. So the likely benefit is perhaps at best 1/10th and potentially <1/100th of what's cited here.
Granted, this is not necessarily obvious stuff. However it is and should be easily understood by anyone holding themselves out to be giving financial advice to other people or institutions.
One quick P.S.: Clearly if our low, 1%, ask were creating moral permission for people to give less than they otherwise would, it would be bad. We don't see much evidence of this - very few of our donors were giving to EA causes before pledging to OFTW, and not many were giving at all.
I'm currently the chair of OFTW's volunteer group so hopefully can help answer this. I'll take your questions together as they're related.
On the current giving as % of income rate: There's a chart of our progress on this over time about half way down the post. This year, about half of donors are giving at least 0.9% of their income.
On average all time the proportion would be lower than this. This is an area where we're making progress by a) better understanding our donor base (what makes them resistant to giving more?) b) better training / equipping our campus leaders to ask their classmates to make a larger pledge (our chapter leader training has materially improved on this front) and c) improving our donation platform settings and defaults.
We have a small (single digit) number of donors who have given meaningfully more than this as a result of OFTW. We've done some small experiments aimed at getting existing donors to increase their pledges, but haven't had much success and haven't prioritized this as much as we have training new chapter leaders and increasing our penetration rates in existing chapters.
Ultimately getting people acculturated to thoughtful, effective giving is the near term goal, because if we build that habit it gives us a platform for bigger asks. If you think about the long term there are two potential prizes: 1) get a generation of people to give to effective causes and to give more, earlier than they otherwise would, and 2) get people to give more effectively on the path they would've otherwise followed (e.g. once someone is mid career and would normally give to their alma mater, give to AMF). Clearly 1) is a much bigger, more valuable change, and we want to work out how to do it. But preserving a relationship and building affinity over time also keeps 2) alive as an option, which is valuable.
Sure:
Cash balance methodology
You say:
Incorrect and misleading statements in these four paragraphs:
In your reply to me you suggest:
Even this approach seems incorrect or misleading:
Potential returns of other investment options
You say:
Citing the one year, backward looking return of JPST is misleading, and citing the one year drawdown performance is naive:
Conclusion and staff time
You say:
I agree with Rob's points.
This article makes several statements that I interpret as somewhere between intentionally misleading and outright incorrect.
For instance, this reply says the 2.19% Fed Funds rate was the prevailing rate in 2019, and isn't intended to be a forecast of the future. But since we know the Fed Funds rate is 0% today, why would that rate be used to argue for a forward-looking change in behavior?
Similarly, the 2019 return of a short-term bond fund gives literally no information about what returns a similar fund would offer today. Indeed the beauty of the bond market is that it does offer guaranteed future returns if you hold a bond to maturity. As of today, US government bonds (which carry no risk of default) offer a guaranteed annualized return of:
So assuming an organization needed to make a grant within 12 moths, they could currently expect to make 0.10% on their cash balance. Needless to say this is 1/10th of 1%, or <1/20th of the figure cited in the post. And it seems a stretch to assume organizations hold cash for 12 months before granting it. So the likely benefit is perhaps at best 1/10th and potentially <1/100th of what's cited here.
Granted, this is not necessarily obvious stuff. However it is and should be easily understood by anyone holding themselves out to be giving financial advice to other people or institutions.
One quick P.S.: Clearly if our low, 1%, ask were creating moral permission for people to give less than they otherwise would, it would be bad. We don't see much evidence of this - very few of our donors were giving to EA causes before pledging to OFTW, and not many were giving at all.
Hi Aaaron,
I'm currently the chair of OFTW's volunteer group so hopefully can help answer this. I'll take your questions together as they're related.
On the current giving as % of income rate: There's a chart of our progress on this over time about half way down the post. This year, about half of donors are giving at least 0.9% of their income.
On average all time the proportion would be lower than this. This is an area where we're making progress by a) better understanding our donor base (what makes them resistant to giving more?) b) better training / equipping our campus leaders to ask their classmates to make a larger pledge (our chapter leader training has materially improved on this front) and c) improving our donation platform settings and defaults.
We have a small (single digit) number of donors who have given meaningfully more than this as a result of OFTW. We've done some small experiments aimed at getting existing donors to increase their pledges, but haven't had much success and haven't prioritized this as much as we have training new chapter leaders and increasing our penetration rates in existing chapters.
Ultimately getting people acculturated to thoughtful, effective giving is the near term goal, because if we build that habit it gives us a platform for bigger asks. If you think about the long term there are two potential prizes: 1) get a generation of people to give to effective causes and to give more, earlier than they otherwise would, and 2) get people to give more effectively on the path they would've otherwise followed (e.g. once someone is mid career and would normally give to their alma mater, give to AMF). Clearly 1) is a much bigger, more valuable change, and we want to work out how to do it. But preserving a relationship and building affinity over time also keeps 2) alive as an option, which is valuable.
Steve