My reflections on 5 criticisms of FarmKind’s bonus system:

Hello!

After receiving impassioned criticisms on our announcement post last week, I decided to use a plane trip (I’ve been on leave) to reflect on them with a scout mindset to make sure Thom and I aren’t missing anything important that would mean we should change our approach. I’m glad I did, because on my way back from leave I noticed this new post. I thought it would help to share my reflections.

To set expectations: We won’t be able to continue engaging in the discussion on this here. This is not because we consider the “case closed”, but because we are a team of 2 running a brand new organization so we need to prioritise how we use our time. It’s important (and a good use of time) for us to make sure we consider criticisms and that we are confident we are doing the right thing.^[1] But there is a limit to how much time we can dedicate to this particular discussion. Please enjoy the ongoing discussion, and apologies that we can’t prioritise further engagement with it :)

Before I get to my reflections, I want to point out an unhelpful equivocation I’ve seen in the discourse: Some of the comments speak of donation matching as if it’s one very specific thing, and they claim or imply that we are doing that one thing. In reality, “donation matching” can refer to a wide range of arrangements that can work in all sorts of ways.

The most common form of donation matching that I’ve seen involves a large individual or institutional donor promising to match all donations to a specific charity 1:1 until a certain date. This can be fully counterfactual (if the large donor chooses to participate in the match where they wouldn’t otherwise have given to that charity or any other) or it can be not counterfactual at all (if they would have given that amount to that charity no matter what), or somewhere in between. What we’re doing is different to this, as we explain here and below, which is why we don’t call it donation matching, as we aren’t wanting people to import expectations about how it works from previous experiences they may have had with common forms of donation matching.

It’s fair to have qualms with how some forms of donation matching work (or even all forms of it, as some commenters do). But most of the reasons given for concern about donation matching apply to some and not all forms of it. We would be able to make better sense of this collectively by being precise about what forms of donation matching we do and don’t have problems with and for what reasons.

Okay, on to my reflections on 5 criticisms of FarmKind’s bonus system:

Criticism 1: It’s not possible to donate $X in total and have >$X go to your Favorite Charity, which means we’re not offering a ‘meaningful match’ / we’re not delivering what we promise

We received this criticism: “The fact that you can't donate $X and get more than $X going to your favourite charity means I don't really feel like my donation is being meaningfully matched”.

We never promise or imply that it would be possible to donate $X in total and have >$X go to your Favorite Charity (after splitting and receiving a bonus). We make it very clear that it won’t:

Our comms make it very clear that FarmKind’s purpose is to help users donate to fix factory farming through our Super-effective Charities, with the ability to support your Favorite Charity at the same time and to get a bonus included as perks. It’s hard to see how anyone could read our comms and see things the way the critics are worried they will, and so it’s hard to see how anyone could be misled:

Criticism 2: Our comms suggest that your Favorite Charity receives money from the Bonus Fund, but that isn’t true

We received this criticism: "The 'bonus' is presented to users as if it (a) will go in part to their favorite charity and (b) is money that would not otherwise be going to help animals, but neither of these are true"^[2]

We think this is incorrect, because it quite literally IS true that the Favorite Charity receives money from the Bonus Fund. This is how the money flows: Every.org splits the regular donors donation between their chosen Favorite and Super-Effective charity and then Every.org disburses money to each from the Bonus Fund in the exact way that’s summarised during the donation process:

Critics have pointed out that the effect of these cash flows is the same as a different set of cash flows where more of the donor’s money was given to their Favorite Charity than indicated (e.g. $90 of the $150 in the donation went to the Favorite Charity), and all of the bonus went to the Super-Effective Charity ($30 in this case). Those critics claim that this is a simpler set of cash flows, and a simpler/clearer way to understand what has happened. We disagree that it’s simpler or clearer, and even if it were, it wouldn’t be what actually happened, and it wouldn’t make our description misleading.

Criticism 3: Our comms are misleading regarding whether the bonuses we add to donations are counterfactual (criticism fleshed out below)

This is an interesting and nuanced one. The criticism goes something like:

The average donor assumes that in donation matching, the matching funding added to their donation would not have otherwise…
- (a) been donated OR
- (b) been donated to the same cause OR
- (c) been donated to the same charities
Because we will not promise a bonus for a donation if the money isn’t already in the Bonus Fund, and if no donations were ever made that money would eventually go to the Impact Fund, our bonus system doesn’t work the way the average donor assumes it does, and we don’t make this clear enough, and so it’s misleading

There are valid aspects of this criticism, which I nonetheless disagree with, and there are invalid aspects. Let’s start with what the criticism gets right and wrong about the reality of the counterfactuality of the bonus system:

Counterfactuality of the bonus system

Regular donations do, in expectation, cause more counterfactual donations to occur. Let’s talk through it, considering counterfactuals in both the narrower/direct and the broader/indirect sense:

1. The narrower/direct sense — i.e. Thinking just about the money that’s already in the bonus fund when a regular donation is made:

(a) Total donations: At the moment that a regular donation is made, that donation doesn’t increase the total amount of money given to charity, because the funds in the Bonus Fund have already been given, and if no more donations ever occurred and FarmKind shut down, we would give the balance of the Bonus Fund to our Super-effective Charities (as explained at farmkind.giving/support-bonus-fund) ❌ no direct counterfactual impact on total donations ❌
(b) Causes: The regular donor gets to pick any Favorite Charity, from any cause, and their donation will cause money from the Bonus Fund to go to it. Unless by some miracle, the Bonus Fund supporters would otherwise have collectively donated to the same causes as the regular donors in the same proportions, then regular donations do have direct counterfactual impact on how much money goes to different causes ✅ direct counterfactual impact on donations to different causes ✅
(c) Charities: The regular donor gets to pick any Favorite Charity and one of our Super-effective Charities and their donation will cause money from the Bonus Fund to go to it. Unless by some miracle, the Bonus Fund supporters would otherwise have collectively donated to the same charities as the regular donors in the same proportions, then regular donations do have direct counterfactual impact on how much money goes to different charities (both ‘effective’ and ‘ineffective’) ✅ direct counterfactual impact on donations to different charities ✅

2. The broader/indirect sense — Regular donations on our platform, which receive a bonus, use up the supply of money that’s in the pool. The more regular donations that occur (i.e. the more demand there is to redeem our bonuses), the more that folks who like how our bonus system is motivation donations will want to keep the system going, by donating to the Bonus Fund. In particular, if the Bonus Fund is running low, we will communicate this to people who like our bonus system, imploring them to donate to keep the system going. As such, each regular donation increases, in expectation, the amount of money contributed to the Bonus Fund.

This is like the how buying one of chickens at the grocery store (which the grocer has already bought and so pre-committed to selling or throwing away) increases the expected amount of chicken they will buy next time. If you think that lower demand for animal products can decrease the amount of animal products produced, you should understand how increased demand for bonus funding can increase the amount of money given to the Bonus Fund.

But what would have happened to the money donated to the Bonus Fund were it not for past demand?

(a) Total donations: That’s what M&E is for, but in all likelihood, as with all fundraising efforts, some money raised would otherwise have gone to charity and some wouldn’t have ✅ indirect counterfactual impact on total donations ✅
(b) Causes: For the money that would have otherwise been given to charity, the odds are infinitesimal that it would have gone to the same causes and specific charities that the regular donors chose, in the same proportions ✅ indirect counterfactual impact on donations to different causes ✅
(c) Charities: See above^ ✅ indirect counterfactual impact on donations to different charities ✅

Do we communicate about it in a dishonest way?

Certainly not. Our communications are accurate, both in the ‘large print’ and the ‘fine print’.

Do we communicate about it in a misleading way?

It is possible to be misleading without being dishonest, however. This can be done by presenting information in a way that leads someone to draw incorrect conclusions, even though the facts themselves are technically true. Typical ways to do this is by omitting certain information, or framing the facts in a particular way.^[3]

To cut right to the chase: Will some people who don’t read our communications carefully think that they’re having direct counterfactual impact on total donations, when they aren’t [as per (1)(a)]? Probably yes, there is always a risk that some people will get the wrong idea. However, we don’t believe that this constitutes misleading communication. Let me explain why:

We take reasonable steps to dispel misconceptions, making what we believe are appropriate and ethical trade-offs to ensure honest communication (although people could reasonably disagree):
- Measures we take:
  - We don’t call what we do “matching”, in order to reduce the association with the most common form of matching campaigns (and common misconceptions that come with them)
  - In order to donate, users must click a box confirming that they agree with our terms of use and that they “understand that the bonus funds are already pre-paid by bonus fund supporters. Learn how our bonus system works here.”
  - If one clicks any of the various links to learn more about how the process works, they’ll be taken to farmkind.giving/support-bonus-fund, where we explain exactly how the system does and doesn’t allow regular and bonus fund donors to increase their counterfactual impact (I won’t quote the explanation here because it’s too long).
  - In the section about the Bonus Fund on our About Us page, we say: “We help you multiply your impact by boosting your donation with prepaid bonus funds… These funds have been contributed by other donors who want to motivate more people to donate to fix factory farming. When you get you a bonus added to your donation through our platform, you get to direct these funds to the Favorite charity and the recommended Super-effective charity of your choosing.”
- The need for trade-offs in honest communication:
  - Honest communication involves trade-offs between two goals (among others): Having those who read it form maximally true beliefs — this is the ‘honest’ bit — and making your communication sufficiently simple, concise and interesting that people actually listen to /read it — without this, you haven’t actually done the ‘communication’ bit.
  - Everyone we’ve ever encountered makes these sorts of trade-offs, and considers them to be a part of honest communication. Of course, it’s also possible to make the trade-off wrong and fail to communicate or do so honestly. Here are various some examples, some of which you may think get the balance right and others of which you may not:
    - Every product/service’s terms and conditions and how they surface them to users (I think we’ve all seen examples we think are acceptable and others we think aren’t. Do any readers think that 100% of cases where one clicks to agree with terms and conditions without being forced to read them in detail are misleading?)
    - Teaching Newtonian Physics to early high-school students, even though it’s technically incorrect (I think this is acceptable, as it’s simple enough for students at this level to understand and is a helpful building block towards the correct model that uses Special Relativity)
    - I've put two more in the footnotes for brevity's sake^[4]
- To sum this point up: People who read the donation form carefully as they fill it out (or who is otherwise curious and so clicks around to learn what this “bonus” thing is all about) will understand it and be making an informed donation decision. People who skim it, may misunderstand what’s happening and so not be fully informed. We’ve taken measures to reduce the likelihood of such misunderstandings, because we think it’s the right thing to do for deontological reasons. It would be possible to take more intense measures (like a series of pop-ups reminding donors several times that bonus funds are pre-committed), but given the trade-offs, we think it wouldn’t be right for consequentialist reasons. Different people will reasonably disagree on what measures we ought to take for deontological reasons, and there is no level that is immune from the criticism that they ought to do more.
Even if donors have a misconception about how participating in the bonus system increases their impact, they understand that it increases impact. As discussed, despite our efforts, a skim reading donor may believe that getting a bonus is counterfactually impactful in a way that it isn’t, but there are still 5 other ways (2 direct, 3 indirect) that it IS counterfactually impactful, causing more donations to go to the things they care about. This, after all, is the promise that makes receiving a bonus attractive.

Criticism 4: By encouraging both regular and bonus donors to participate in the bonus system to amplify their impact, we’re causing a double-counting of impact which is suboptimal for collective resource allocation

This is also an interesting one, which we’re grateful was raised, and we will incorporate this into future comms.

Such a double count is likely, and is one of the challenges with counting impact using standard counterfactual reasoning (see discussion here and here). These kinds of double-countings are actually really common. For example, whenever two advocacy groups both play a necessary role in achieving a policy change, they will generally both conclude that the counterfactual impact of their work is the full impact of the policy change. This double count of impact can lead to an inaccurate view of the cost-effectiveness of each group, leading to suboptimal grantmaking decisions.

What are we to do about this? Surely the answer isn’t to not encourage both groups to participate in our bonus system — after all, the participation of each group counterfactually causes higher impact. Each group benefits (in terms of increased impact. We don’t think the answer is to introduce Shapley Values or some other more complicated means of impact attribution, which would compromise our ability to communicate the value to each group. Rather, we should probably talk about the increased impact each group will have in general terms, without using specific numbers unless someone asks for them and/or we heavily caveat them. Having received this criticism, we would not use specific numbers the way we did here again.

It’s worth mentioning that, of course, resource allocation is extremely far from perfectly efficient, so the real cost of these kinds of double counts is probably very low. It’s also worth reiterating that this problem applies to nearly every intervention in EA — it is not a particular problem of donation matching.

Criticism 5: FarmKind uses a form of donation matching, and even if fully understood and consented to by all parties, matching does not belong in EA because.. [various reasons]

It is reasonable to disagree about what is and is not ‘EA’. For example, we have similar disagreements with many EAs’ cause area prioritisation, epistemics, use of funding, communications styles. We also see many projects that don’t meet what we see as minimum standards in terms of the existence of feedback loops, theories of change, or measurement and evaluation.

We personally wouldn’t choose to claim there is no place for these people or projects in EA. We think that there is value in a more pluralistic and diverse EA community that takes many different approaches, all motivated by the honest desire to do the most good we can.

I’m sure it’s clear from our choice to launch this platform that we think this is a good thing to be doing, and that it’s our attempt at doing altruism effectively. If some folks who disagree decide that FarmKind does not belong in effective altruism, there is likely nothing we can say that will change their mind. That’s ok.

I don’t have more to say about this criticism except to reiterate that I respect its intention, I disagree and I thank people for sharing it. We considered Jeff’s critique post of Giving Multiplier prior to deciding to launch FarmKind and we were grateful to come across this perspective before rather than after making the decision — it influenced decisions we made for the better.

Thanks for reading, and happy foruming!

^{^}
To that end, if any new points are made in response that we haven’t considered, we will consider them and may change our mind, but we may not neccessarily report back about this.
^{^}
Criticism 2 is about part (a) of this statement. We discuss part (b) under Criticism 3
^{^}
For example, stating "Our product has been used by over 1 million people!", when 99% of people used it once, were dissatisfied and never used it again. The statement is true (and in that sense, not dishonest), but it omits context so that it might lead someone to believe that 1 million people are active, happy users, which isn't the case.
^{^}
3rd example: The donation platform Ribon using the half-sentence “Ensure free-range living for farm animals” to describe the impact of donating to a specific charity, even though no given donation can possibly guarantee this outcome is achieved (I think this is acceptable, and I’d hope that further comms were available for those who are interested enough to read more to understand how their donations might have an impact).
4th example: Saying “I like ice cream”, even if you think that the self is an illusion (an illusion which your statement reinforces), such that you’re giving the false impression that you think the self is a real thing (I include this example because I imagine that even our most steadfast critics would agree this is honest communication. The kind of person who caveats their use of the word ‘I’ is probably failing to communicate)

EA Forum Bot Site
EA Forum

Aidan Alexander

Posts
7

Comments
56

My reflections on 5 criticisms of FarmKind’s bonus system:

Criticism 1: It’s not possible to donate $X in total and have >$X go to your Favorite Charity, which means we’re not offering a ‘meaningful match’ / we’re not delivering what we promise

Criticism 2: Our comms suggest that your Favorite Charity receives money from the Bonus Fund, but that isn’t true

Criticism 3: Our comms are misleading regarding whether the bonuses we add to donations are counterfactual (criticism fleshed out below)

Counterfactuality of the bonus system

Do we communicate about it in a dishonest way?

Do we communicate about it in a misleading way?

Criticism 4: By encouraging both regular and bonus donors to participate in the bonus system to amplify their impact, we’re causing a double-counting of impact which is suboptimal for collective resource allocation

Criticism 5: FarmKind uses a form of donation matching, and even if fully understood and consented to by all parties, matching does not belong in EA because.. [various reasons]

Aidan Alexander

Posts 7

Comments56

My reflections on 5 criticisms of FarmKind’s bonus system:

Criticism 1: It’s not possible to donate $X in total and have >$X go to your Favorite Charity, which means we’re not offering a ‘meaningful match’ / we’re not delivering what we promise

Criticism 2: Our comms suggest that your Favorite Charity receives money from the Bonus Fund, but that isn’t true

Criticism 3: Our comms are misleading regarding whether the bonuses we add to donations are counterfactual (criticism fleshed out below)

Counterfactuality of the bonus system

Do we communicate about it in a dishonest way?

Do we communicate about it in a misleading way?

Criticism 4: By encouraging both regular and bonus donors to participate in the bonus system to amplify their impact, we’re causing a double-counting of impact which is suboptimal for collective resource allocation

Criticism 5: FarmKind uses a form of donation matching, and even if fully understood and consented to by all parties, matching does not belong in EA because.. [various reasons]

Posts
7

Comments
56