ML

Matt_Lerner

1659 karmaJoined

Bio

Currently Research Director at Founders Pledge, but posts and comments represent my own opinions, not FP’s, unless otherwise noted.

I worked previously as a data scientist and as a journalist.

Comments
127

Thanks! This is very helpful/informative — particularly the thing about YouTube!

To be 100% clear about what I see as the main issue (by what I think are 80k's lights), it's not that the podcast is less interesting for an EA Forum audience, but rather that it's less interesting in general. It's a niche podcast for people who already think AI is very important.

I'm sort of confused by how this interacts with the goals laid out in the Google Doc. I think it's great to target elite decision-makers — but I would have assumed the greatest impact is on the extensive margin, among people who (1) have decision-making power but aren't AI specialists or (2) don't already have well-developed views.

By not offering content that will allow the podcast to grow along this margin, I would worry that you are preaching to various existing choirs! I certainly can't imagine anyone becoming interested in working in AI as a result of this podcast - they'll never listen!

But I think surely you have thought about this — I am interested in the answer, though.

No, I really don't. Sometimes you see things in the same territory on Dwarkesh (which is very AI-focused) or Econtalk (which is shorter and less and less interesting to me lately). Rationally Speaking was wonderful but appears to be done. Hear This Idea is intermittent and often more narrowly focused. You get similar guests on podcasts like Jolly Swagman but the discussion is often at too low of a level, with worse questions asked. I have little hope of finding episodes like those with Hannah Ritchie, Christopher Brown, Andy Weber, or Glen Weyl anywhere else anytime soon. It's actually a big loss in my life and (IMO) leaving many future potential EAs and AI people on the table.

It's great to see the podcast expanding. I think the ship has already sailed on this, but it feels important for me to flag two experiences I've had since the podcast's "shift to AI."

  1. I listen much less than I used to. This is partly because I end up thinking plenty about AI at work, but also because podcasts occupy a middle ground between entertainment and informativeness for me. Though I think AI is critically important, it is not something I get a real kick out of thinking and hearing about.
  2. I share episodes with non-EAs much less than I used to. Most normies I know are sick of hearing about AI and, moreover, there's no longer any content to engage people who don't want to listen to a three-hour podcast about AI. I think that's a shame, since many of those people would have happily listened to a three-hour podcast about e.g. vaccines, subscribed, and then learned about AI at a later date.

While this is true, I think this comment somewhat misunderstands the point of the post, or at least doesn't engage with the most promising interpretation of it.

I work at Founders Pledge, and I do think it is true that the correct function of an org like FP is (speaking roughly) to move money that is not effectiveness-focused from "bad stuff" to "good stuff." Over the years, FP has had many conversations about the extent to which we want to encourage "better" giving as opposed to "best" giving. I think we've basically cohered on the view that focusing on "better" giving (within e.g. American education or arts giving or whatever) is a kind of value drift for FP that would reduce our impact over the medium- and long-term. We try to move money to the best stuff, not to better stuff.

But I still think this is a promising frame that deserves some dedicated attention.

Imagine two interventions, neither of which is cost-effective by "EA lights." Intervention A can save a life for $100,000. Intervention B can save a life for $1 million. Enormous amounts of money are spent on Intervention B, which is popular among normie donors for reasons that are unrelated to effectiveness. If you can shift $1 billion from Intervention B to Intervention A, you save 9,000 lives. Thus working to shift this money is cost-effective — competitive with GiveWell top charities in expectation — if it costs less than $45 million to do so (~$5,000 per life).

More generally, shifting money by ~0.1 GDx is cost-effective even if you're shifting money far below the cost-effectiveness bar as long as you have 100x leverage over the money you are shifting.

I don't think opportunities like this are super easy to find, but it seems plausible to me that the following avenues will contain promising options:

  • Simply working in a position with very high leverage over funds moved (e.g. in government or at a major normie philanthropy)
  • Promoting effective interventions in areas where (a) lots of money is spent and (b) the dominant intervention is not effective
  • Value of information interventions that have a clear effect on decision-making
  • Some kinds of journalism

FP Research Director here.

I think Aidan and the GWWC team did a very thorough job on their evaluation, and in some respects I think the report serves a valuable function in pushing us towards various kinds of process improvements.

I also understand why GWWC came to the decision they did: to not recommend GHDF as competitive with GiveWell. But I'm also skeptical that any organization other than GiveWell could pass this bar in GHD, since it seems that in the context of the evaluation GiveWell constitutes not just a benchmark for point-estimate CEAs but also a benchmark for various kinds of evaluation practices and levels of certainty.

I think this comes through in three key differences in perspective:

  1. Can a grant only be identified as cost-effective in expectation if lots of time is spent making an unbiased, precise estimate of its cost-effectiveness?
  2. Should CEAs be the singular determinant of whether or not a grant gets made?
  3. Is maximizing calculated EV in the case of each individual grant the best way to ensure cost-effectiveness over the span of an entire grantmaking programme?

My claim is that, although I'm fairly sure sure GWWC would not explicitly say "yes" to each of these questions, the implication of their approach suggests otherwise. FP, meanwhile, thinks the answer to each is clearly "no." I should say that GWWC has been quite open in saying that they think GHDF could pass the bar or might even pass it today — but I share other commenters' skepticism that this could be true by GWWC's lights in the context of the report! Obviously, though, we at FP think the GHDF is >10x.

The GHDF is risk-neutral. Consequently, we think that spending time reducing uncertainty about small grants is not worthwhile: it trades off against time that could be spent evaluating and making more plausibly high-EV grants. As Rosie notes in her comment, a principal function of the GHDF has been to provide urgent stopgap funding to organizations that quite often end up actually receiving funding from GW. Spending GW-tier effort getting more certain about $50k-$200k grants literally means that we don't spend that time evaluating new high-EV opportunities. If these organizations die or fail to grow quickly, we miss out on potentially huge upside of the kind that we see in other orgs of which FP has been an early supporter. Rosie lists several such organizations in her comment.

The time and effort that we don't spend matching GiveWell's time expenditure results in higher variance around our EV estimates, and one component of that variance is indeed human error. We should reduce that error rate — but the existence of mistakes isn't prima facie evidence of lack of rigor. In our view, the rigor lies in optimizing our processes to maximize EV over the long-term. This is why we have, for instance, guidelines for time expenditure based on the counterfactual value of researcher time. This programme entails some tolerance for error. I don't think this is special pleading: you can look at GHDF's list of grantees and find a good number that we identified as cost-effective before having that analysis corroborated by later analysis from GiveWell or other donors. This historical giving record, in combination with GWWC's analysis, is what I think prospective GHDF donors should use to decide whether or not to give to the Fund.

Finally - a common (and IMO reasonable) criticism of EA-aligned or EA-adjacent organizations is an undue focus on quantification: "looking under the lamppost." We want to avoid this without becoming detached from the base numeric truth, so one particular way we want to avoid it is by allowing difficult-to-quantify considerations to tilt us toward or away from a prospective grant. We do CEAs in nearly every case, but for the GHDF they serve an indicative purpose (as they often do at e.g. Open Phil) rather than a determinative one (as they often do at e.g. GiveWell). Non-quantitative considerations are elaborated and assessed in our internal recommendation template, which GWWC had access to but which I feel they somewhat underweighted in their analysis. These kinds of considerations find their way into our CEAs as well, particularly in the form of subjective inputs that GWWC, for their part, found unjustified.

Hey Darren, thanks for doing this AMA — and thanks for doing your part to steer money to such a valuably and critically important cause.

Can you describe a bit about the decision-making process at Beast Philanthropy? More to the point, what would an optimal decision-making process look like, in your view? e.g. how would you use research, how would you balance giving locally vs globally, think about doing the most good possible (or constraining that in some way), etc?

I listened to the whole episode — if I understood correctly, they are mostly skeptical that there are effects at very low blood lead levels. At the end of the podcast, Stuart or Tom (can't remember which) explicitly says that they're not skeptical that lead affects IQ, and they spend most of the episode addressing the claimed relationship at low BLLs (rather than the high ones addressed by LEEP, CGD, other interventions).

I'd be interested in exploring funding this and the broader question of ensuring funding stability and security robustness for critical OS infrastructure. @Peter Wildeford is this something you guys are considering looking at?

I'm also strongly interested in this research topic — note that although the problem is worst in the U.S., the availability and affordability of fentanyl (which appears to be driving OD deaths) suggests that this could easily spread to LMICs in the medium-term, suggesting that preventive measures such as vaccines could even be cost-effective by traditional metrics.

Load more