Estimating the Average Impact of an ARPA-E Grantmaker

charrin

Comments 8

Sorted by

New & upvoted

"2. Secondly, we assume that all benefits -- including health benefits, reduction of monetary costs associated with climate change, reduction of existential risk associated with climate change, and spillover benefits into other industries -- are incorporated into the market valuation. We believe that markets likely price in the aforementioned externalities (e.g., health and enviro benefits) (support for this claim here). Furthermore, the valuations listed on ARPA-E’s site are from after the Inflation Reduction Act passed, which, itself, internalized a significant chunk of US emissions. For these reasons, we believe it’s reasonable to assume a significant amount of external benefits have been internalized into markets, but, perhaps, not all benefits. Thus, we believe that, all else equal, this assumption leads to an underestimation of benefits. "

Thanks for this article! The above assumption feels quite wrong to me and, as such, I expect this makes your estimate a vast underestimate everything else being equal.

You seem to assume that climate risk and other externalities are priced in market valuations. Even if that were true for the US (which seems unlikely) it would certainly not be true for most places in the world that have very little in terms of pricing energy-related externalities. Given the role of US in the global energy innovation system, it seems reasonable to assume that the benefits that ARPA-E creates are way larger than market valuations, at least if market valuations are -- as you suggest -- reflective of expected policy returns.

ethai

Yeah, strong agree with this. [I used to work in VC and frequently diligenced ARPA-E grantees.] I don't think the cited study supports the claim that all externalities are priced in in the US, let alone globally.

I would also guess that the valuation of the 26 exited companies is an underestimation of overall impact for other reasons - top of mind: impact of non-exited companies, learning benefit to the field of a company that "fails" and exits at a very low valuation.

@charrin thanks for writing this, as a below commenter said it's nice to see an EA-style investigation of a potentially impactful career path outside the community!

charrin

Appreciate the feedback. We definitely agree that not all benefits are captured in market valuation and that this leads to an underestimate. While working on this, we initially tried creating an "external benefits multiplier,” but ultimately couldn’t come up with a particularly sound way of estimating this multiplier prior to the submission deadline. Regardless, I think we were not focused enough on the fact that the US plays this significant role in the global energy innovation system, so thanks for bringing that up too. In retrospect, we should’ve just added something as a placeholder and then made updates later.

The general framework we could use to create this multiplier is to calculate the ratio of a standard carbon price recommended by environmental economists (e.g., ~$51 - 125 per ton) over an estimate of the “implicit US carbon price” (i.e., standard carbon price/implicit US carbon price). The implicit US carbon price could be derived by calculating an emissions-weighted carbon price associated with RGGI and the California cap-and-trade market, as well as the implied dollar paid per emission abated through current US subsidies for carbon reduction technologies. We could rely on work from UChicago’s EPIC to help guess the implied dollar paid per emission abated through US subsidies. However, as you can see in the aforementioned UChicago paper, US policies likely pay way more than $125/ton in some instances (e.g., with the weatherization assistance program) and miss cheap emissions reduction opportunities in other instances (e.g., nature-based solutions). This could make the “implicit US carbon price” look artificially high. Regardless, the implied emissions price may still be relatively low (e.g., <$30/ton). Thus, the external benefits multiplier might be something like 2 or 3.

If we further account for the US’ role in the global innovation system, potential existential risks associated with climate change, etc., we could increase the numerator.

Would love either of your thoughts on this approach!

Ivy Mazzola

Cool work! Props that you allow for people using their own discount rate, as your first footnote is a good point.

I think that for transparency and ease of reader understanding, you ought to link the 80K's article on grantmaking for most-pressing problems. Similarly, linking info on Squiggle would be good.

Also, best to clarify that this review is only about "working at a government agency that funds relevant research", and that this is only one out of their 3 mentioned highly-effective careers related to grantmaking (at the bottom), and that the other two would need a different analysis.

I also think that if the intent is to advise on careers, you need to do some analysis of the team of E-ARPA. Variables that come to mind are: size of team, how many of each role, how senior each person seems (for thoughts on how soon a person can get hired in such a role, and maybe even, in the case of junior employees, where they could go from there career-capital-wise), and a rough guesstimate of how much each role contributed to the overall annual grantmaking decisions of E-ARPA.

Also a minor wording note.. You say:
"We chose ARPA-E because other ARPA agencies are explicitly called out by 80,000 Hours profile of grantmaking (i.e., DARPA, IARPA)"

"Called out" has negative connotations, so I'd probably say "mentioned", "referred to", or "brought up" instead. That terminology confused me--I thought you were saying you only chose ARPA-E because the others had been essentially ruled out. I was a bit aghast thinking you'd chosen an example in a category where others had already been shown to be moot or something , and that's why I dug for and read the original 80K piece >.>

Phew, sorry that was so much seemingly-critical feedback, but to clarify I (not a researcher or data scientist) think what you did do is good and I'm happy you reviewed this career path, which tends to, I think, be unfortunately skipped in many career discussions. I strong upvoted the post.