~11min read

Introduction

This post provides an overview and analysis of the Doing Good Better  book giveaway through Effective Altruism New Zealand (EANZ). The analysis covers data collected from survey responses between 05-Jan-17 and 17-Dec-19, for which there were a total of 298 responses, with appreciable variance in the amount of the survey which was completed. This analysis was initially completed around Jan 2020 so any reference to "to date" refers to then. 

All monetary values are in NZD. 
Tables with specific data are reported in the appendices at the end of the post. 

Epistemic Status: I am not highly confident about specific values, for reasons of self-reporting on behalf of the respondents, and some assumptions on the analysis side.

Overview

The book giveaway is accessed through an online form on the EANZ website. The online form did not ask specific questions on donations - so part of the analysis had to be done by comparing survey results with donation data from the EANZ Charitable Trust (which forwards tax-deductible donations to a select number of GiveWell top charities).

Data was collected in two ways:

1. Tracking book recipients who started donating to the EANZCT after receiving the book.
2. Book recipients were emailed an anonymous survey, 3 months after being sent a book.

Key Outcomes

• Book recipients on average donated $28 to the EANZCT in the first year after they received the book. Deducting the costs of the book & delivery ($15), this accounts for a net increase of $13 in the first year, and $38 in the first two years. This is likely to be an underestimation as people may be donating through other channels or to other charities, and it doesn’t track donations from people who received the book from their friends.
• Survey respondents reported they would give $232  per year more to effective charities as a result of reading the book. This equates to $56 per book, per year. However, these numbers have a high uncertainty as: a) non-respondents may also still donate, b) social-desirability bias may be inflating numbers, and c) people may not actually donate what they said they will.
• 29% of respondents reported that reading the book made them reconsider their career path, and 30% their dietary choices.

The main suggestion to increase the accuracy of the data and of a future analysis is to have a more thorough pre-giveaway survey that includes similar questions as the follow-up survey. This would decrease the need to compare different sources of data and reduces reliance on donors who donate to effective charities via the EANZCT. Surveying first-time EANZCT donors may also provide valuable data.

Survey Questions & Results

The follow-up survey was sent via email to giveaway participants approximately 3 months after being sent a book. Bracketed numbers on graphs indicate total responses to the question.

Section 1 - Pre-book Data

The first section of the survey assessed the knowledge, beliefs and donations of the participants before they were sent the book.

Prior EA Knowledge

As we would expect, the majority of people who received the book had heard little to nothing of EA before applying for the giveaway.

EA Affiliation

As expected, the majority of people didn’t consider themselves an EA prior to the book. 13% (38) of respondents said they considered themselves to be an effective altruist. 67% (196) responded no, and 20% (57) responded maybe.

Surprisingly, there were 8 people who didn’t know what effective altruism was, yet still considered themselves to be EA’s. The table below shows the responses of those who answered “Yes” to considering themselves an effective altruist before the book. It appears there isn’t a strong correlation between how much someone knew about EA prior to the book and whether they consider themselves an effective altruist. Some respondents chose both “I didn’t know what effective altruism was” and “I had been to effective altruism meetups”, which is particularly peculiar.

However, for those who said they “maybe” considered themselves an EA before the book (57), they mostly either didn’t know what it was or had heard of it but didn’t know much.

Prior Donations

The largest number of respondents donated between $100 and $999 prior to reading the book. The overall total was $342,000, an average of $1,200 and a median annual donation of $100 (both including the $0 responses). 
 

The donations according to how the respondents identified with EA are shown in the table below. On average, the more someone identified with EA, the higher their annual donations to charity, which is nice to see.

Another question asked in this section was: "If you donated to charities in the past, what charities did you support ? (Give up to four charities)". This data was used to inform the later analysis on increased funds directed to effective charities. (See Appendix B for calculations)

On average, 10% of the charities which respondents donated to were effective charities, prior to reading the book.
 

Section 2 - Post-book data

The second section of the survey asks respondents if they have read the book, and assesses behaviour/belief changes of those which have read at least part of it.  

Reading completion

About half of respondents had read the book, with 84% (247) having read at least part of it. These respondents were asked follow up questions about behaviour change. Those who said they had not read the book were not asked follow up questions.
 

Donation influences

63% (149) of respondents said reading the book influenced their charity choices, and 48% (114) said they would donate more. Respondents were asked how much more or less they would donate, depending on their response. Of the 114 respondents who said they would donate more, 94 said they would donate more than $0. Overall the result was a 26% increase in planned annual donations due to reading the book. Three respondents stated they would donate less, one of which ($2000 less) was due to "increasing volunteering hours in their local community and to develop start-up social enterprises", and another due to donating more time.

Note: This is general charitable donations, not specifically donations to effective charities.

On average, 34% of the charities which respondents said they donate to now (post-book), or intend to donate to in the future, were effective charities.
 

EA Affiliation

31% (70) respondents now consider themselves part of the EA community as a result of reading the book, with 45% (101) unsure and only 9% (20) responding no. 

Career & diet changes

Respondents were asked whether or not they were reconsidering their career path, or diet, as a result of reading the book. 29% of respondents said that reading the book made them reconsider their career path, and 30% would reconsider their dietary choices. 14% (32) respondents said that reading the book made them reconsider both their career path and dietary choices.

Section 3 - EA Influence

In the final section, respondents were asked about whether they had introduced others to EA ideas, and if so, how they responded to these.

Of 245 respondents, the total number of people they had lent/given the book to was 227, meaning that on average each person passed the book on to (just less than) one person. However, as a result of reading the book, 222 respondents reported that they had introduced a total of 1363 people to EA ideas, with a mean of 6 people and a median of 3 people. The majority (139, 68%) of 205 respondents who had introduced people to EA ideas said that the response to the ideas was positive. One respondent reported that there was a negative response.
 

Key Takeaways

Expected/reported changes to effective donations

The respondents who already donated to effective charities, and would continue to do so, said they would donate an overall $23,700 more to effective charities (per year), with an average of $1,820 and a median of $250 (per respondent per year). Those who donated to ineffective charities and switched to effective charities would reportedly donate a total of $45,200 more to effective charities, with an average of $900 and a median of $440. This brings an expected total of $68,900 more going to effective charities, which accounts for 78.75% of the total increase in donations.

• With 297 respondents to the survey, this results in an average $232 increase in expected funds going to effective charities per respondent.
• With 1244 books sent, this equates to an average $56 increase in expected funds going to effective charities per book.

Notably, one individual reported they would donate an estimated $660 less to effective charites due to “reduc[ing] my working hours to do more volunteer work in my local community and work to develop new start-up social enterprises”.

Donors to EANZ Charitable Trust

In order to calculate the 'actual' cost-effectiveness of each book given out, the amount donated to the EANZCT by donors who made their first donation after receiving the book was investigated. It was found that within one year of receiving the book, the total amount donated was approx. $35,000, equating to $28 per book sent. The donations were also summed to date and this was found to equate to $53 per book given away. This provides a strong case for the effectiveness of the giveaway of the Doing Good Better books. 

Considering the cost per book including postage is $20, it was found that the net donation to effective charities per book per year was $8, and $33 to date! Evidently this isn’t highly aligned with the expected increase in donations based on the survey, however it is possible donors made donations through channels other than the EANZCT. Perhaps the survey can be used as a proxy for actual donations. 

Noteworthy free-responses

Respondents were given the opportunity to make additional comments at the end of the survey. Here are some noteworthy ones (they mainly focus on the book itself, so feel free to skip):

- "The book was well worth reading, glad I picked it up, it will change the way I go about giving in the future, in effort to make a better difference"

- "Book giveaway is a good idea, in my opinion. Many people I have spoken to cite the book as the thing that got them initially involved in EA - this survey may say the same but from personal experience, many of those who are most involved got involved at least partially due to the book."

- "I read the book straight away and found it very informative. I usually get all my books from the library, so wouldn't have bought it if it wasn't free. You have reminded me to get the book back from the person I loaned it to and give it to someone else."

- "I think giving the book away was a good idea. Based on it's cover / title, I would not have purchased the book."

- "It was very good so far but my brain started to hurt and I had to take a break, have lent to a friend and will get it back later. Perhaps a "for dummies" version could be good?"

One other point of note is that several people mentioned that they wish there was more coverage on the effectiveness of NZ charities. This seems to be a recurring theme when introducing people to EA here in NZ (and I am uncertain what should be done about it, if anything at all).

Key Improvements

• Include a question for first-time EANZCT donors to ask if their donation was a result the book giveaway
• Expand the pre-giveaway data collection to include questions about donations, and specifically ask in both the pre- and post- giveaway survey about current donations rather than planned/expected ones
• Include a dropdown list of the major charities, with an "other" option (would save a LOT of time on analysis)
• At the end of the survey, ask for contact details of respondents and whether they would be ok with us following up with them regarding their responses.
  • Some individuals mentioned they would be interested in participating in events / learning more - however we had no means to reach out to them.

Other ideas

• Develop a system to include book identifier numbers that go with the book e.g. sticker. This will support better tracking and follow-up systems for when the book is passed on - essentially increasing the impact of each book (or at minimum allowing for better data capture).
• Potentially include questions around demographic data, though I'm uncertain how valuable this would be

Final Thoughts

I think the book giveaway is a worthwhile endeavor - especially as word of EA grows worldwide. It provides people with a deeper dive into EA topics and introduces the core concepts. Furthermore, the data indicates the giveaway directs more funds to effective charities than the costs to run the giveaway, which, coupled with the other effects (awareness of EA, potential career trajectory changes, etc) suggests the giveaway is effective. With some minor adjustments and improvements to the system/process - I am interested to see the results in a year or two's time. 

If anyone would like to start a book giveaway, has suggestions on how ours could be improved, or has other thoughts - I'd love to hear about it!

Appendices (Google Doc)

Appendix A: Specific response data per question
Appendix B: Calculation of effective donations

Edit: Amended book + delivery cost to $20, from $15.