Our very crude evaluation:

We expect that EA Madison has reached something like 50 person-hours of attention (excluding Gina and Ben), though that could easily be off by a factor of 2 in either direction and we have talked to no one else involved with it.

Our very rough guess for the stimulated donations per hour was around $20. A lot of this comes in the form of general engagement rather than directly in the form of donations. This gives a value of about $1000 of donations stimulated.

We expect it to be possible to make a much less noisy evaluation in the future once we can evaluate some of the impact of the group in retrospect. We think our evaluation will predictably rise in expectation over time, reflecting our general preference for purchasing things after their impacts are easier to assess. Our actual expectation for the project's impact obviously can't predictably rise.

Good idea, done.

Our very rough evaluation:

We looked at the number of upvotes on LessWrong and the list of people who Alexei thought might have benefited during their own job searches. We contacted a handful of these people and asked about how much the post helped them. The sample was not random, and Alexei's list was clearly not exhaustive.

Our impression is that the largest benefits were from:

• Gaining confidence and pursuing a career search in a more wholehearted way.
• Increasing the number of interviews that people took, which probably improved their offers and decreased time unemployed.
• Modestly improving people's ability to perform well on interviews
• Modestly improving people's willingness to negotiate.

Most of these effects have somewhat ambiguous signs, i.e. people made tradeoffs differently and it's not obvious that they made improvements. Although we have some reservations with some of the advice in this post, the biggest effects seemed to be improving people's understanding of the recruitment process, and we expect that effect to be positive.

Our extremely rough estimate was that this effect amounted to $200k in total increased (pre-tax) earnings. We guess that on the order of 5-15 people benefited significantly from the post, that the per individual benefits were on the order of $5-10k / year, and that these benefits probably persist anywhere from one to a few years (in many cases the benefit was finding a similarly good job faster, which seems to have a comparably large impact concentrated in one year). This is the crux of our estimate, but it isn't very well informed.

Based on the individuals who benefited we'd guess that about 10% of this money is used to cost-effectively make the future better (and probably somewhat more), and we are valuing the intervention at about $20k EA dollars moved.

We are generally inclined to count financial benefits beyond the fraction that is donated, and our 10% reduction is not motivated entirely by this estimate for donations (which we think is a bit low, and which we haven't adjusted to account for the fact that beneficiaries were disproportionately likely to support causes that we like rather than other EA causes). In this case there are a number of considerations that justify the discount to 10%:

• Much of the salary gains reflect zero sum competition amongst applicants or between applicants and employers, though there are some increases in allocative efficiency.
• Much of the credit for any positive impact of the advice is reserved for the people who actually made changes on its basis.
• Related to both of the above, to the extent that taking more interviews, being more likely to pursue a job, and negotiating more aggressively create value, they also have costs that claw back some of the benefits.

I think this case was probably my biggest disagreement with Paul; I thought this project had quite high value. As it happened I didn't end up purchasing any, presumably because the gap between

• My evaluation of the HPMOR wrap parties
• The seller's evaluation of the HPMOR wrap parties

was larger than the gap between

• My evaluation of the thesis
• The seller's evaluation of the thesis

but I would like to signal that I am willing in general to bid non-trivial amounts for academic work on X-risk and value drift.

The technique I used to value it was to estimate how long it would take MIRI to produce such a piece, how valuable it was compared to MIRI's typical output, and how much it costs MIRI to hire researchers.

The fact it was a doctoral thesis did cause me to assign a significant discount to my valuation. I'm not really sure how to think about this.

This was another project that was very hard to evaluate. Overall, we were sufficiently uncertain and skeptical about the content of the chapter that we thought it was very unlikely that we would have a higher (honest) valuation than the author, and we didn't make a more detailed evaluation.

Our very crude evaluation:

This was pretty tough to compare to the direct EA work, and it was a small project. We did a ridiculous evaluation just to see how it turned out.

We tried to make the comparison by thinking about how many dollars of EA donations would be required to achieve a comparably good outcome (according to our values). To think about this we considered the "scaled up" version of the activity that reached essentially all US youth from a single year and was repeated enough to have a substantial effect on their education and view of computer science, which we imagined was about 2 million times (or 10 such sessions per student). We then compared that impact to the effect of targeted funding in the areas of science with the most leverage and altruistic impact. We guesstimated that the impact would be about 1/100k the impact of an EA grant the size of the annual budget for US R&D and science (and change) which is something like $1T. (The 100k came from multiplying estimates for the impact of marginal STEM education on research quality/enthusiasm/etc., the relative importance of research quality vs. funding, the extra bang-for-your-buck by targeting the best areas and spending money effectively. We chose the annual US budget because the scaled up intervention reached 1 year of students in the US. The impact on marginal STEM education takes into account the fact that the intervention is just 10 sessions.)

This all suggests a value of something like $5 of stimulated EA donations, which we wouldn't take too seriously :)

(In case it's not clear, we don't endorse this procedure for prioritizing very different causes.)

Our crude evaluation:

We think the EA community in Melbourne has had a significant impact, with many people either making large donations or working on EA projects full time who would not have done so otherwise. We're evaluating this impact at around $100k / year, even considering only donations+replacement costs at EA organizations, which we expect to significantly understate the real impact.

Many of these effects seem likely to be long-lasting, and we feel comfortable extending the benefits over at least 5 years.

It is much harder to attribute these impacts to the formal EA Melbourne group, as compared to the informal community, the LW group, TLYCS, the actions of individual EA's in Melbourne, and so on. We had a few conversations to try to get a better sense of this allocation of responsibility. In the end we certainly didn't get a confident answer, but we got a vague intuitive feeling for the situation.

Based on this impression, we allocated 10% of the responsibility to the formal organization of EA Melbourne. Interpreted narrowly we think this is more likely to be an overestimate than an underestimate.

But we think that there are other benefits from EA Melbourne that can justify this estimate, and which will tend to be on the same scale as the direct effect. For example, the broader EA community in Melbourne clearly had a positive effect; but EA Melbourne looks like it will have a big impact building and growing a similar community in the future. And to the extent that other organized communities and the online presence of EA played a big role, it seems that EA Melbourne has made similar contributions back to the broader EA community.

Our estimates concern the impact of EA Melbourne during its first 6 months. We assumed that Brayden and Ryan were almost entirely responsible for the founding of EA Melbourne. People other than Brayden and Ryan were clearly involved with the operation of EA Melbourne over this period. Conversely, EA Melbourne continued to exist after this initial period and it seems likely that its founding will continue to have positive impacts going forward beyond those already mentioned. In the end we called this a wash.

Overall, we evaluated EA Melbourne at $70k in stimulated EA donations.

We purchased 1/70th of this in each of rounds 2 and 3 for a total of $2700. Owen Cotton-Barratt purchased 0.4% in round 3 for $200.

The original certificate was divided evenly between Brayden and Ryan.

Current holdings:

• Brayden: 48.37%
• Ryan: 48.37%
• Impact purchase: 2.87%
• Owen: 0.4%

We purchase 3% in round 1 for $400, which we resold to Larks for $300 in round 3. Larks purchased 6.9% for $700 in round 3.

Current holdings:

• Oliver: 90.1%
• Larks: 9.9%

Our very rough evaluation:

• We estimate that around 1300 people went to wrap parties (adjusted somewhat for how long they were there for). This was based on examining the list of events and their purported attendances, and a few quick checks for verification.
• We estimated Oliver’s impact was 1/4 of the impact of the wrap parties. We estimated that the existence of cental organization doubled the scale of the event, and we attributed half of that credit to the central organization and half of the credit to other local organizers and non-organizational inputs (which also had to scale up).
• We estimated that the attendance of an additional person was worth around $15 of stimulated EA donations. This was a guess based on a few different lines of reasoning. We estimated the value of the EA/LW community in stimulated donations, the value of annual growth, the fraction of that growth that comes from outreach (as opposed to improving the EA product, or natural social contact), and the fraction of outreach that came from the wrap parties. We also guessed what fraction of people were new, would become more involved in the EA/LW community as a result, and how much value that would create. We sanity checked these numbers against the kind of value participants probably got from the celebration individually.
• So we have 1300 * 15 /4 = $4,875 of stimulated EA donations, which we rounded up to $5,000.

Our very crude evaluation:

We had a hard time estimating the total impact of this research. It laid out an interesting case that modafinil is a reasonable alternative to caffeine, at least setting aside social factors. It did not seem to credibly address the main empirical questions that would motivate us to adopt either modafinil or caffeine, and we expect that most readers would be similarly skeptical (if they were sufficiently open-minded to plausibly take modafinil on the basis of analysis). We thought about evaluation by considering how hard it would be to produce a similar amount of value by paying for empirical research or critical review that would help clarify the benefits of modafinil and potentially push adoption.

It was most convenient to measure other submissions in terms of a metric like "dollars of EA donations stimulated," so to make our decision we tried to convert "dollars moved to GiveDirectly" to this metric.

Even though many EA donations will go to GiveDirectly or other GiveWell charities, there were a few reasons we used a discount:

• "EA donations stimulated" come with other significant advantages.
• Although many EA donations go to development, some of them go to other causes like capacity-building, GCR reduction, or AI. We are significantly more excited about these other donations.

We're not sure exactly what the multiplier should be, especially in light of positive institutional and cultural effects from funding effective work in development. But we felt comfortable using at least a factor of 5 discount, which we think is large enough that none of the development interventions are likely to be competitive unless they are extremely leveraged.

We plan to think about the multiplier more if we receive submissions that seem like they might be competitive anyway.

We didn't really want to evaluate or purchase this item ourselves, because it seems too self-serving / meta.

Is the number of reads really relevant? How come? I figure people who read content generally don't act on it, and certainly not in high impact ways.

We purchased 1/2 of this post in round 1 for $600. We have since sold 1/3 of the post to Owen Cotton-Barratt for $400. Current holdings:

• Ben: 1/2
• Owen: 1/3
• Impact purchase: 1/6

Our very crude evaluation:

• We estimate that EAs and other related groups move around $500k annually through donation matching. We are thinking of drives run by MIRI, CFAR, GiveDirectly, Charity Science, Good Ventures, among others.
• We think a full and clear understanding of donation matching would improve this by around $6k, through such drives being better optimized. We lowered this figure to account for the data being less relevant to some matching drives, and costs and inefficiencies in the information being spread.
• We think this work constitutes around 1/30th of a full and clear understanding of donation matching.
• We used a time horizon of three years, though in retrospect it probably should have been longer. This implicitly included some general concerns about the fraction of people who have seen it being smaller in the future, and information accruing from other sources and conditions changing, and so on.
• We get $6,000 * 3 years / 30 = $600 of stimulated EA donations

In addition to this analysis we did some sanity-checking based on readership and the plausible direct impact of the post on smaller matching drives. Overall this seemed like a plausible estimate to us.