Grantmakers: Consider sharing rough probabilities & brief feedback with grantees/applicants

Some thoughts based on some personal experience,^[1] something I think could make people and  organizations who depend on grant funding are more effective. Maybe particularly salient now, given the fast pace of AI development. Much of this also applies to employers as well as to ongoing funding relationships, not just grants.  

Costs of applying, incentives to invest in polish

Funding calls and grant applications impose substantial costs on applicants, cutting into the total effective value of the resources that ~"EA can deploy". A lot of work goes into prepping applications, making them look good, and making strategic and stylistic adjustments that aren't always aligned with adding value to the project, the ecosystem, or the world. The worst case is something like an all-pay auction, where in equilibrium the applicants between them invest something close to the full value of the grant in preparing their applications.^[2]

Of course, there are also costs on the other side: it can take a lot of time for funders to process and adjudicate these applications.   

I think the EA grantmaking ecosystem does an OK job of holding costs down: short forms, word limits, applications presented as "just give us a first picture of what you do and why." But there's an unavoidable minimum-friction floor when substantial money and personal interest are at stake. Even a mostly effective-altruism-aligned and rationalist applicant has some personal interest, and also may tend to overvalue their own cause and approach relative to a perfect Bayesian (~winner's curse behavior). So it makes sense, from a private PoV, to tweak and optimize extensively.^[3] 

Planning and calibrating 

There's a second aspect here. Once you've submitted, or while you're working through a grant and waiting to hear about renewal, you don't know whether you're far above the bar, far below it, or close to it. I made this point in a comment a couple of years ago: applicants may waste a lot of time exploring a very dark space. They don't know when to give up, how much to build fallback plans, how much to adjust the proposal, or in what direction.

A lot of this cost is in planning. If funding is very unlikely, the right move is to get on with the small, low-budget version of the work right now. If funding is likely, that same effort is better spent preparing to move quickly at scale: contractors, staff, compute. With short timelines on AI risk, and salient, actionable, high-impact work to be done right now this matters more. 

The cost of this uncertainty can particularly impact coordination and/or movement building. The people around an organization (volunteers, contractors, allies. etc).  who are deciding how much to engage with it are deciding blind too, and in spaces with several overlapping initiatives, nobody can tell which ones to coalesce around.

It also imposes avoidable costs on the funders and grantmakers themselves. ~The less certainty a nonprofit has about grant funding, the more likely they are to apply for additional grants, sometimes multiple streams from the same granter, which must be processed and adjudicated. Caveat:^[4]

This applies beyond the period when an application is pending, which, to be fair, can be reasonably short if planned well. It applies at least as much to organizations you're already funding. Over the course of a grant, I imagine funders form and revise their views: what's going well, what they're having doubts about, how likely they are to renew. Grantees mostly don't see any of that until the renewal decision. If there were some light in this darkness, this would help the grantee prioritize and plan during the grant.  I'd love to asee a place a grantee could check for the funder's current assessment , an updated probability of further funding.

Also high-value: some notes on the things the funder is uncertain about and would like explained or justified better – this  can make aan eventual renewal application more concise and more useful to the funder too.

Funders could  consider sharing 

... both while an application is pending and during a grant:

A rough probability. "We currently predict a 65% chance you get this at your median request or higher." This can be given explicitly as a quick impression, explicitly noisy.   It could even be an AI's take based on context that is not shared with the grantee; tell  an AI or agent something like\

Check our notes and correspondence on the XYZ grant and predict the probability we renew it, and the distribution of funding in expectation. Do a scheduled update every 2 weeks, 
1. Post it on the following password-protected (or anonymized) Grantee Checkin page which we will share with the (potential) grantee.
2.Update us with your rationale for  that in our "Grantee portfolio space" but do not share the latter outside our organization.
In the latter space, provide your best take on what's most in doubt about the application/organization, and prompt us to decide whether to share this in the Grantee Checkin page as well   

Again, I think grantees want to know, in particular, 

• Their probability of success at different funding levels  
  • And it's OK to say "you're well below the bar and this looks like a mismatch with our goals", ideally providing at least some justification and epistemic basis (but not necessary)
• What's convincing and what's in doubt, to be able to focus their communication work, and their actual practical work. ~"We're convinced about this part of what you do. We'd want to hear more about that part."
• Grant decision timelines, and whether things are being delayed

The cost-benefit of doing this

This is not free, and I see some reasonable objections. Still, I think offering quick and course private signals has a pretty good benefit-cost here.

Linch highlights the time and opportunity costs --  I think these costs are substantially lower using AI tools. He hints at emotional and political issues getting in the way, and some fear that applicants would overupdate on feedback based on fast takes: I suspect that most EA-aligned grantees/applicants are not too vulnerable to this. And it's ameliorated by carefully generated epistemic status. "This was a probability generated by AI based on our unshared notes" also seems much less likely to lead to overupdating. 

Theere are other issues involving information cascades: these mainly only apply if the feedback is made public or widely shared, not if it's only shared with applicants and grantees.  (Okay, I'd be a bit careful with this:  a specific number like "30% likely to pass our bar" is perhaps more portable and legible, so perhaps more likely to leak to other funders and cause these negative cascades.)

I suspect legal issues are less important here.^[5] 

Costs don't seem terribly high to me^[6].

On the stated precision: I'd rather funders gave their best guess at a number and made clear it's rough than withheld it. People in this ecosystem are used to estimates like "67%, low confidence" by now, and I doubt many will mistake one for a precise, calibrated forecast. But if buckets (likely, unclear, unlikely) feel more comfortable, that's still much better than nothing. Giving a base success rate also helps. 

Anyone doing this/willing to try it?

Curious: To what extent are funders doin this? Would any funder be willing to try the explicit forecast part? I think even a probability plus an expected decision date, sent at a fixed point after submission, and a light quantitative check-in and steering in the middle of a grant period seems better than silence.  I'd be glad to help design or evaluate a pilot, and perhaps it's something I could help trial and facilitate in context related to my work. 

1. ^{^}

   Mostly experience on the grantee and applicant side, but indirectly in the context of being an employer and contractor.  

2. ^{^}

   I let AI remind me/check about the game theoretic and actual game theory results and economics experiment lab evidence :

   The all-pay auction is the right analogy rather than the war of attrition: both are contests where losers don't get their effort back, but in a war of attrition everyone pays the losing bid, since the contest stops when the weaker party concedes, whereas in an all-pay auction each player pays their own bid. Krishna and Morgan (1997) describe them as second-price and first-price all-pay auctions respectively. Grant applications are the first-price case — nobody gets a refund when a rival gives up. 

   On magnitudes: in the standard complete-information models, expected total expenditure can't exceed the prize, since a player who can bid zero must earn non-negative expected profit, and summing that over players bounds total spending by the prize value. Baye, Kovenock and de Vries (1994) prove this for the Tullock contest at every returns-to-scale parameter, and attribute earlier over-dissipation claims to non-equilibrium solutions.

   What does exceed the prize is realized spending: the same authors (1999) show the probability of ex post over-dissipation is one half with two players in a perfectly discriminating contest, falling to about 0.44 as the number of players grows. Experimentally it's worse than the theory — Dechenaux, Kovenock and Sheremeta's survey reports total effort "often twice to five times higher than the common prize value," and Sheremeta puts the median overbidding rate at 72%. 

   Two caveats in the other direction. First, the full-dissipation result belongs to the symmetric mixed equilibrium; the complete-information war of attrition also has asymmetric pure-strategy equilibria that burn nothing (Hendricks, Weiss and Wilson 1988), and Georgiadis, Kim and Kwon (2022) argue the mixed equilibrium doesn't survive adding noise to payoffs. Second, with asymmetric valuations the strongest contestant keeps positive rents and dissipation is incomplete, bounded by the second-highest valuation. Applicants to a given fund are very heterogeneous, so I'd expect real dissipation well below 100% — which is why I say "something like" and don't lean on the equilibrium result. ↩

3. ^{^}

   As an applicant, you want to get under the word count while still putting your best foot forward, in a way that will appeal to a range of possible readers. Even very short forms can eat days, more so when several organizations or team members are involved and have different views about what will work. And refining applications down to the precise right word or character count might be a fun exercise for wordsmiths, but it's a time-consuming one.

^{^}

There may be cases in which sharing more information leads nonprofits to apply for more grants, e.g., if they were initially confident of 95% success, and the information moves them to 50%. I'll think more  about the equilibrium expected  effect of this.

^{^}

In employment consequences there are legal issues to take into account, like getting sued or discrimination, with 'anything you share with applicants used against you' My weak take is that even these concerns are a bit overblown, and I suspect this is less important  in grantmaking space.

^{^}

Fable suggests:
> Funders already forecast. Open Philanthropy has made thousands of explicit probabilistic predictions during grant investigations and scored them. The Survival and Flourishing Fund asks applicants how likely they think their other fundraising is to succeed, and asks them to be quantitative about their own confidence.  The US National Institutes of Health gives every applicant a percentile and written critiques within about a month.

Show all footnotes