Linda Linsefors recently wrote a post making the case for impact purchases. I've always been somewhat skeptical of impact purchases, at least as they are currently applied. I see two key problems with impact purchases: firstly that the market of sellers is far larger than the market of buyers and secondly that many people would have done those projects anyway.
Buyer's Market vs. Seller's Market
How much impact do these purchases have on your decision to pursue a project? If you think you are almost certainly going to be funded, then it'll make a huge difference. If it is only a small chance, then it'll make almost no difference at all. The problem I saw in a past impact purchase (can't find the link) was that there were way too many people trying to sell impact with all of them chasing a vanishingly small amount of money. This meant that projects either had a small chance of having their impact purchased or that the amount of money provided was small compared to the effort that went into the project.
One potential solution to this is to limit the size of the market. In the impact purchase I saw, the buyer was willing to consider almost anything. While this ensured that high-value projects weren't excluded from consideration, it also made the imbalance between the impact for sale and available capital as large as possible. For this reason, I'd suggest defining a narrow market - for example AI safety or wild animal suffering only.
However, the market might still be far larger than the available capital. One way to limit it further would be to limit the purchase to a certain group of people. For example you might only include alumni of Harvard or people who attended an AI safety camp. This results in the exclusion of a lot of talent, however it's arguably better to motivate a few talented people a lot than a much larger pool a little. You could limit it further by only considering projects that occurred in a certain time period.
The Counterfactual
Regardless of whether you use an impact purchase or provide funding normally, there's always a chance that the project would have taken place regardless. However, impact purchases greatly increase this problem since the person already did the project without knowing whether or not they would be funded.
Let's imagine a near optimal scenario where if the project succeeds you have a 90% chance of being funded. Even a 10% chance of not be funded or receiving minimal funding could be a deal-breaker in many circumstances. But of course, this doesn't include the chance of the project failing. And considering the market dynamics above, the chance of the project being funded reasonably could be extremely small. This tends to suggests that the projects that get funded might only be those that would have and could have occurred anyway. As mentioned before, narrowing the field might make incentives strong enough for more of the difference to be counterfactual.
Another issue is that only people who knew about the impact purchase during a project could have been influenced by it. Limiting it by category of person can help here, as say if it's only open to alumni of program X, the organisers of that program might be able to contact those eligible to let them now. Another possibility would be to make anyone eligible if they register as potentially interested during a particular period. Further, if we limit it to, say, the next two years, then we exclude all projects that were completely before the announcement and which hence couldn't have been influenced.
One possible response is that even if these projects aren't counterfactual, people who create impact deserve funding as a matter of fairness. This seems like a reasonable position, but notice that it is in tension with utilitarianism. Another argument is that people who had an impact in the past are likely to spend the money to have an impact in the future. This might be the case, but if this is the primary vehicle through which these purchases have an impact, it might be worthwhile crafting a mechanism that more narrowly focuses on this.
Other issues
Another issue with impact purchases is the potential to harm relations between people, lead to bitterness or to demotivate people. People tend to be very personally invested in their projects and tend to have quite a high evaluation of the impact of their projects. Buyers will often only value projects at a fraction of what the seller thinks that it is worth. This may lead to the seller feeling unappreciated or insulted. This is compounded if the seller put a lot of time and energy into the project with the expectation of some kind of reimbursement. Even though people are rejected from grants as well, there's a difference between someone deciding another project is more promising and someone explicitly telling you that your project isn't worth anywhere near what you think it is. I don't have a suggested solution to this problem.
I agree. This is not a crux for me.
I currently think "people who had an impact in the past are likely to spend the money to have an impact in the future", is the main argument for Impact Purchase. It is possible that impact purchase is not the optimal format. I am still thinking about this. But I think it is important that "look at all the stuff I did in the past" is *enough* to get funded, no explanation of what I will do next needed, becasue that is too inflexible (see my post).
And if we are going to trust people and hand them money based on what they did in the past, it does make very much sense to me to trust them to the extent of how much good they done in the past. We want to give our trust and money to people who are competent (can successfully complete their plans) and have good judgement (have a good idea of which projects are potentially very important). Impact tracks both those metric.
When making a guess, one should start with the outside view, and then adjust from there. In most cases, the best outside view for what a person will do in the future, is what have they done in the past. Then maybe we want to do some adjustment from there?
If a person seem very reckless, maybe don't fund them. Or if you think an outcome was mostly bad luck, fund them more than just impact purchase. But in most cases I would suggest straight up impact purchase, because anything else is is really hard and you'll probably get the adjustments wrong.
This is already happening. People do start things in the hope of getting funding later along the way. And it is not just about projects. There is a reason posts complaining about how EA treats people gets very upvoted.
I think everything would be much better if we stopped worrying so much and started treating people as adults.
If a person does something that is good, but not good enough to be worth paying for, than this means that EA rather have money than this work. This means that if this person want to maximise impact, they should find something better to do, or up-skill, or switch to earning to give. Under most circumstances they should not keep doing that thing, so we should not encourage it.
(People are of course allowed to things that are less than optimal, if this is what they want. I am very much in favour of people doing what they want. But we should not pretend that what they do is more important than it is, just for encouragement.)
I went to my first EAG in 2016. Earlier that year I had finished my physics PhD and found out about AI Safety. At EAG almost everyone I talked to encouraged me to retrain to do AI Safety, and I felt super motivated. But I was running out of saving, so I asked for some money to take me through the transition, and most people just thought that I was weird for asking. That was demotivation as hell. These people where encouraging me to use my last savings to retrain to a risky career, but putting in their money was out of the question. This told me that they considerer spending my time and my resources to be costless. I was seen as a tool to be used, not an allay.
(Things have gotten better for me after that, but this is still a painful memory. And to be honest, I am still a bit bitter about it.)
People are not dumb. If encouragement is not backed up by action or money, they will notice.
If there is not enough money to go around to pay reasonable salaries for everyone who does important work, then this means that we a money constrained, which means that we would be better of if some of these people should switch to earning to give. If this is the case we as a community should be upfront about this. People will understand and adjust.
The way to not antagonise people is to be upfront about everything.
"These people where encouraging me to use my last savings to retrain to a risky career, but putting in their money was out of the question" - Yeah, I'm sorry you had that experience, that seems unpleasant. Maybe they didn't understand the financial precariousness of your situation? Like many EAs wouldn't find it hard to land a cushy job in normal times and likely have parents who'd bail them out worst comes to worst and might have assumed that you were in the same position without realising they were making an assumption?
I did get help from my parents later, so if that where their assumptions they where not wrong. But I did not know this at the time, and when I asked why I could not get funding I got answers of the type: "that is not how things are done", which made no sense to me.
It is possible that not funding me back then where the right decision for the right reason. But since I where not told the reason, the experience for me where very discouraging and antagonising. That's why transparency is important!
(I'm not really blaming anyone. I think that the people I where talking to did not have explicit knowledge, so where therefore not even able to answer me. But I think we can do better.)
Yeah, luck is another argument I considered covering, but didn't get into. Sometimes the impact of a project is just a matter of being at the right place in the right time. Of course, it's hard to tell; to a certain extent people make their own luck.
I guess this would be a key point where we differ. I haven't thought deeply about this, but my intuition would be that adjustments would greatly improve impact. For example, a small project extremely competently implemented and a big project poorly implemented might have the exact same impact, but the former would be a stronger signal.
In this case, the competent person can just do more great small projects and get more money.