Some concerns about policy work funding and the Long Term Future Fund

FWIW, I think this kind of questioning is fairly Habryka-specific and not really standard for our policy applicants; I think in many cases I wouldn’t expect that it would lead to productive discussions (and in fact could be counterproductive, in that it might put off potential allies who we might want to work with later).

I make the calls on who is the primary evaluator for which grants; as Habryka said, I think he is probably most skeptical of policy work among people on the LTFF, and hasn’t been the primary evaluator for almost any (maybe none?) of the policy-related grants we’ve had. In your case, I thought it was unusually likely that a discussion between you and Habryka would be productive and helpful for my evaluation of the grant (though I was interested primarily in different but related questions, not “whether policy work as a whole is competitive with other grants”), because I generally expect people more embedded in the community (and in the case above, you (Sam) in particular, which I really appreciate), to be more open to pretty frank discussions about the effectiveness of particular plans, lines of work, etc.

I guess it depends how specific you are being. Obviously I don’t think it should be taken as given that "specific think tank plan x" would be good, but I do think it is reasonable for a fund to take it as given that at a high level "policy work" would be good. And if the LTFF does not think this then why does the LTFF actively outreach to policy  people to get them to apply?

I don't think we should take it as a given! I view figuring out questions like this as most of our job, so of course I don't want us to have an institutional commitment to a certain answer in this domain.

And if the LTFF does not think this then why does the LTFF actively outreach to policy  people to get them to apply?

In order to believe that something could potentially be furthered by someone, or that it has potential, I don't think I have to take it as a given that work in that general area "would be good". 

I also think it's important to notice that the LTFF page only lists "policy research" and "advocacy", and doesn't explicitly list "policy advocacy" or "policy work" more broadly (see Asya's clarification below). I don't think we currently actively solicit a lot of policy work for the LTFF, though maybe other fund managers who are more optimistic about that type of work have done more soliciting. 

And separately, the page of course reflects something much closer to the overall funds view (probably with a slant towards Asya, since she is head of the LTFF), and this is generally true for our outreach, and I think it's good and valuable to have people with a diversity of views on the LTFF (and for people who are more skeptical of certain work to talk to the relevant grantees).

Now it is my turn to claim a strawman.  I have never applied to the LTFF with a plan anything close to a "let's try to get smarter people into government" approach. Nor were any of the 5 applications to the LTFF I am aware of anything like this approach.  

Sorry! Seems like this is just me communicating badly. I did not intend to imply (though I can now clearly see how one might read it as such) that your work in-particular falls into this category. I was trying to give some general reasons for why I am skeptical of a lot of policy work (I think only some of these reasons apply to your work). I apologize for the bad communication here.

This strikes me as an odd statement. If you're talking about the LTF fund, or EA long-termism, it doesn't seem like much policy work has been funded.

I think this is false, e.g. a reasonable subset of Open Phil's Transformative AI risks grantmaking is on policy. 

This strikes me as an odd statement. If you're talking about the LTF fund, or EA long-termism, it doesn't seem like much policy work has been funded.

Huh, why do you think that? CSET was Open Phil's largest grant to date, and I know of at least another $20MM+ in policy projects that have been funded. 

Sadly, I think a lot of policy grants are announced less publicly, because publicity is usually harmful for policy projects or positions (which I think is at least some evidence of them being at least somewhat adversarial/non-cooperative, which is one of the reasons why I have a somewhat higher prior against policy projects). Approximately all policy applications to the LTFF end up requesting that we do not publish a public writeup on them, so we often refer them to private funders if we think they are a good idea. 

If you're talking more broadly, wouldn't policy wins like decreasing the amount of lead being emitted into the atmosphere (which has negative effects on IQ and health generally) be a big policy win for the long term future?

Yeah, I think this is definitely a candidate for a great intervention, though I think importantly it wasn't the result of someone entering the policy space with a longtermist mindset. 

If someone had a concrete policy they wanted to push for (or a plan for discovering policies) of that magnitude, then I would likely be excited about funding it, though I would still be somewhat worried how likely it would be to differentially accelerate development of dangerous technologies vs. increase humanities ability to navigate rapid technological change (since most risk to the future is anthropogenic, I am generally skeptical of interventions that just speed up technological progress across the board), but my sense is abating lead poisoning looks better than most other things on this dimension.

Oh, sorry, I didn't intend this at all as a critique. I intended this as a way to communicate that I don't think I am that alone in thinking that most policy projects are pretty unlikely to be helpful.

I strongly agree with Sam on the first point regarding downside risks. My view, based on a range of separate but similar interactions with EA funders, is that they tend to overrate the risks of accidental harm ^[1] from policy projects, and especially so for more entrepreneurial, early-stage efforts.

To back this up a bit, let's take a closer look at the risk factors Asya cited in the comment above. 

• Pushing policies that are harmful. In any institutional context where policy decisions matter, there is a huge ecosystem of existing players, ranging from industry lobbyists to funders to media outlets to think tanks to agency staff to policymakers themselves, who are also trying to influence the outcomes of legislation/regulation/etc. in their preferred direction. As a result, making policies actually become reality is inherently quite hard and almost impossible to make happen without achieving buy-in from a diverse range of stakeholders. While that process can be frustrating and often results in watering down really good ideas to something less inspiring, it is actually quite good for mitigating the downside risks from bad policies! It's understandable to think of such a volatile mix of influences as scary and something to be avoided, but we should also consider the possibility that it is a productive way to stress-test ideas coming out of EA/longtermist communities by exposing them to audiences with different interests and perspectives. After all, these interests at least in part do reflect the landscape of competing motivations and goals in the public more generally, and thus are often relevant for whether a policy idea will be successful or not.
• Making key issues partisan. My view is that this is much more likely to happen by way of involvement in electoral politics than traditional policy-advocacy work. Importantly, though, we just had a high-profile test of this idea in the form of Carrick Flynn's bid for Congress. By the logic of EA grantmakers worried about partisan politicization, my sense is that the Flynn campaign is one of the riskiest things this community has ever taken on (and remember, we only saw the primary -- if he had won and run in the general, many Republican politicians' and campaign strategists' first exposure to EA and longtermism would have been by way of seeing a Democrat supported by two of the largest Democratic donors running on EA themes in a competitive race against one of their own.) And yet as it turned out, it did not result in longtermism being politicized basically at all. So while the jury is still out, perhaps a reasonable working hypothesis based on what we've seen thus far is that "try to do good and help people" is just not a very polarizing POV for most people, and therefore we should stress out about it a little less.
• Creating an impression (among policymakers or the broader world) that people who care about the long-term future are offputting, unrealistic, incompetent, or otherwise undesirable to work with. I think this one is pretty easily avoided. If you have someone leading a policy initiative who is any of those things, they probably aren't going to make much progress and their work thus won't cause much harm (other than wasting the grantmaker's money). Furthermore, the increasing media coverage of longtermism and the fact that longtermism has credible allies in society (multiple billionaires, an increasing number of public intellectuals, etc.) both significantly mitigate against the concern expressed here, as the former factors are much more likely to influence a broad set of policymakers' opinions and actions.
• “Taking up the space” such that future actors who want to make long-term future-focused asks are encouraged or expected to work through or coordinate with the existing project. This seems to be more of a general concern about grantmaking to early-stage organizations and doesn't strike me as unique to the policy space at all. If anything, it seems to rest on a questionable premise that there is only one channel for communicating with policymakers and only one organization or individual can occupy that channel at a time. As I stated earlier, policymakers already have huge ecosystems of people trying to influence policy outcomes; another entrant into the mix isn't going to take up much space at all. But also, policymakers themselves are part of a huge bureaucratic apparatus and there are many, many potential levers and points of access that can't all possibly be covered by a single organization. I do agree that coordination is important and desirable, but we shouldn't let in itself that be a barrier to policy entrepreneurship, IMHO.

To be clear, I do think these risks are all real and worth thinking about! But to my reasonably well-informed understanding of at least three EA grantmakers' processes, most of these projects are not judged by way of a sober risk analysis that clearly articulates specific threat models, assigns probabilities to each, and weighs the resulting estimates of harm against a similarly detailed model of the potential benefits. Instead, the risks are assessed on a holistic and qualitative basis, with the result that many things that seem potentially risky are not invested in even if the upside of them working out could really be quite valuable. Furthermore, the risks of not acting are almost never assessed -- if you aren't trying to get the policymaker's attention tomorrow, who's going to get their ear instead, and how likely might it be that it's someone you'd really prefer they didn't listen to?

While there are always going to be applications that are not worth funding in any grantmaking process, I think when it comes to policy and related work we are too ready to let perfect be the enemy of the good.

1. ^{^}

   Important to note that the observations here are most relevant to policymaking in Western democracies; the considerations in other contexts are very different.

and that if you have a range of experts with veto power and extremely different views then perhaps between them every possible idea is vetoed

Just to be clear, the LTFF generally has a philosophy of funding things if at least one fund member is excited about them, and a high bar for a fund member to veto a grant that someone else is championing, so I don't think "every possible idea getting vetoed" is a very likely failure mode of the LTFF. 

Ah, no worries. Are there any new grantmakers with policy backgrounds?

I basically agree with this. On 1, undergrad degrees aren’t a great proxy but particularly the people listed on the LTFF site are all career engineers. On 2, your description sounds like the correct general case, but in a case where non-policy people are questioning the effectiveness of any policy work on the grounds that policy is ineffective, I would expect people who’d worked on it to usually have a brighter view given that they’ve chosen to work on it. 3 is of course up for debate and the main question.)