We would love to get critical feedback (read: free labor) on our evaluation of EA Melbourne, and/or on the comparison to our past evaluations. In the future we may post more projects in the comments and ask for input into their evaluations. Do people think that is worthwhile?
It might put people off submitting projects - I think one ought to ask for permission in these cases.
That's fair. Right now the application says:
""" Some parts of this application may be published and discussed publicly. We will encourage respectful discussion, but there may nevertheless be critical comments. Do you give us permission to publish the sections "The project" and "Optional information"?
The contents may also be the subject of arguments, and may be evaluated erratically or indefensibly. """
Many people marked part or all of their application as private, and we would of course respect that. But it may still be worth asking more specifically.
I'd definitely like to see more insight into what you didn't choose to fund and why.
Thanks for posting these updates, I'm quite excited about the project!
Have you considered incentive problems stemming from the fact that you require fractions of impact to be allocated among participants so that they add up to 1? My understanding is that this way of allocating credit doesn't produce the desired results in cases where the project wouldn't have happened without all participants (see e.g. 5 mistakes of moral reasoning).
If you've already answered this, I'd appreciate a link -- I know you've thought about this quite a bit.
Discussed briefly here.
There are two things going on in these cases:
Impact purchases incorrectly value good deeds based on the highest bidder, instead of summing over all people. They are supposed to work correctly on the supply side, but not the demand side. This is a complicated issue I may discuss later. In order to get a correct protocol you need to combine them with another idea. For the rest of the post I am going to assume that everyone has the same values, which makes this issue go away. Note that this issue is similar for normal donations.
If there are increasing or diminishing returns to scale, the sum of people's marginal contributions don't add up to 1. The simplest case is when output = (sum of inputs)^x for some x other than 1.
If there are decreasing returns to scale, then there are rents: the sum of the marginal outputs adds up to less than the total output, and so there is some extra value to be captured. Certificate purchases work fine in that case---each contributor can unilaterally claim their impact, or the group can claim its impact and decide how to split the rent.
Increasing returns are more complicated. It's still fine to pay a project for its total impact---there is guaranteed to be some way of assigning that impact which would incentivize people to do the project (othertise they should have all done their second-best option, and they would have produced more value that way). Our approach is to group tightly complementary contributions together and let them negotiate a solution. This is the same thing that we normally do in the broader market. Philanthropy normally dodges the issue by just not thinking about it.
Note that the naive strategy of just paying each person for their marginal contribution would also go wrong.
For example, suppose that there are two people, who can each contribute up to 1 unit of effort in a project that creates E^2 value, where E is the amount of effort. Each person can also use their unit of effort to create 2.5 units of value.
It's easy to see that each person's marginal contribution is 4-1 = 3, but in fact they would be better served by both doing the 2.5 thing.
I think that a sufficiently careful accounting of marginal impacts would work, but that is a more complicated issue and I don't have a link to a good account.
Thanks! This reply makes sense to me, and the refutation of the marginal-contribution strategy is interesting. I can see why you've chosen to group tightly complementary contributions.
Round 2 of the impact purchase is over. At the deadline, we had twelve submissions.
This round, we are buying a certificate for 1/70th Ryan Carey and Brayden McLean's founding of and involvement in EA Melbourne during 2013, for $1000.
The deadline for applications to round 3 is May 25th. Apply here!
Below is our evaluation of EA Melbourne. As usual, this is a very rough and quick evaluation. No one involved with EA Melbourne has endorsed this as an account of their impact, and we expect that this evaluation would change significantly if we put in more time.
We think the EA community in Melbourne has had a significant impact, with many people either making large donations or working on EA projects full time who would not have done so otherwise. We're evaluating this impact at around $100k / year, even considering only donations+replacement costs at EA organizations, which we expect to significantly understate the real impact.
Many of these effects seem likely to be long-lasting, and we feel comfortable extending the benefits over at least 5 years.
It is much harder to attribute these impacts to the formal EA Melbourne group, as compared to the informal community, the LW group, TLYCS, the actions of individual EA's in Melbourne, and so on. We had a few conversations to try to get a better sense of this allocation of responsibility. In the end we certainly didn't get a confident answer, but we got a vague intuitive feeling for the situation.
Based on this impression, we allocated 10% of the responsibility to the formal organization of EA Melbourne. Interpreted narrowly we think this is more likely to be an overestimate than an underestimate.
But we think that there are other benefits from EA Melbourne that can justify this estimate, and which will tend to be on the same scale as the direct effect. For example, the broader EA community in Melbourne clearly had a positive effect; but EA Melbourne looks like it will have a big impact building and growing a similar community in the future. And to the extent that other organized communities and the online presence of EA played a big role, it seems that EA Melbourne has made similar contributions back to the broader EA community.
Our estimates concern the impact of EA Melbourne during its first 6 months. We assumed that Brayden and Ryan were almost entirely responsible for the founding of EA Melbourne. People other than Brayden and Ryan were clearly involved with the operation of EA Melbourne over this period. Conversely, EA Melbourne continued to exist after this initial period and it seems likely that its founding will continue to have positive impacts going forward beyond those already mentioned. In the end we called this a wash.
Overall, we evaluated EA Melbourne at $70k in stimulated EA donations. In this round, we purchased impact at a rate of one dollar per dollar of stimulated EA donations, for a total price of $70k. This was about half of the price paid in the previous round.
We are more concerned with future generations than with current suffering, and have valued "dollars moved to GiveWell top charities" several times lower than "EA donations stimulated." We welcome submissions with both kinds of benefits, but wanted to explain why they have not been purchased so far. We do believe that donations to GiveWell's top charities have positive direct and indirect effects on the future, and we also value reductions in current suffering. But these benefits seem substantially smaller than the direct and indirect benefits of more focused interventions, or of organic growth of the EA movement that generates additional donations.
Some submissions involved impacts that have not yet materialized, for example positive effects on people who haven't yet done anything differently as a result, but who may do something differently in the future. For the most part, we have held off on purchasing these.
We continue to evaluate influencing people at a discount based on an uncertain allocation of responsibility. But many of our submissions have been for EA outreach, and they have been the cheapest submissions despite this discount.
Thanks for this Paul and Katja. This will give me marginal financial stability that will help me to self-study machine learning and prepare my next career step!
Echoing Ryan's comments, I'll add thanks for helping fund continuing growth as I re-invest these funds in the San Francisco EA community.