The example you gave is about marginal cost-effectiveness (we spend "$1 on ads"). I agree that then, in this abstract/idealized case, you should spend the $1 on ads. I think all the uncertainty you would realistically have makes it less obvious, though.
But average cost-effectiveness would be more like, we spent $1,000,000 on an organization that did a bunch of different activities, and we think that led to $1,500,000 counterfactually going to charity. This seems good on average, but there's a further question of whether we should give another $1 to the organization. And I think that the 6x figure of the orignal post is referring to average cost-effectiveness ("our current grantees deliver an average adjusted return on donations of 6x across our effective giving portfolio"). This is at least conceptually coherent with the bar for the marginal $ being closer to 1x.
I think you might find the GWWC impact evals interesting, they go into an enormous amount of depth on all these issues.
If it's average future that still could justify a 1x bar, depending on what we're averaging over.
I don't think it does. It's conceptually coherent for an organization to have a very high average cost-effectiveness while also having a marginal cost-effectiveness below 1x. For this reason, I don't think you should have a "bar" for average cost-effectiveness. (You might be making the point that if the average cost-effectiveness is above 1, then you are better off making the grant than burning the money, and so it clears a bar in that sense, but it's not clear it's worth making the grant vs making a potentially much smaller grant, and so it's not a helpful 'bar' in the sense the term is usually used.)
I agree with the concerns about uncertainty, displacing less-effective charities, and counterfactuality. But I'd rather see attempts to adjust the estimate for that rather than ~"we're saying 6x but not really, probably lower after considering this". This will help avoid temptations towards soldier/promotion mentality, and make it more comparable to other estimates.
Sure, but these are hard to account for. I agree it's better to adjust the model when it's possible, but you'll still be left with a model that has a tonne of uncertainty.
(RE "opportunity cost of the labor of the employees who could otherwise do impactful work or eartn to give, etc" -- if EA people are putting in free labor into these efforts, that should also be factored into the cost estimates, naturally, not just the direct CG investment.)
Yep! I wasn't trying to suggest you shouldn't account for that.
Saying 'deliver' to me present tense implies "deliver and will continue to deliver", suggesting the marginal returns should be comparable.
This could still be (and I'd guess is?) referring to the past and expected future average cost-effectiveness.
I also think that it'd be pretty reasonable to have a bar higher than 1x. (I don't know what CG's bar actually is.) There are many contentious choices you make when coming up with a multiplier — e.g., how do you discount future donations, how do you discount donations to less cost-effective charities, do you adjust for the opportunity cost of the labor of the employees who could otherwise do impactful work or eartn to give, etc. There's also just a huge amount of uncertainty in various places, especially around counterfactuality. So given all that, I think it'z reasonable to just zoom out and think: hmm, this intervention looks great overall, but I think the multiplier model isn't robust enough to justify further support of organizations that it estimates only have a 1.1x multiplier.
Curious what you think of the Giving What We Can impact evaluation? I helped write the first one, which claimed that over the 2020–2022 period it had an average multiplier of 30x. I think the more recent one claimed it had a 6x multiplier.
If an investment in an organization can yield anything over 1x in counterfactual donations, this would be worth funding.
I think there's an important distinction between average and marginal effectiveness — the above claim seems true in the abstract for orgs that can move >$1 on the margin but not on average. And IMO it's much harder to estimate marginal cost-effectiveness, because it's forward looking, whereas historical average cost-effectiveness is a bit easier.
Thanks for the kind words :) and appreciate the concreteness of these suggestions.
I think many of these seem to have a common thread of: treating money as a valuable resource, viewing profligacy as costly (both financially and to the soul), seeing signals of dedication and altruism as especially valuable, and generally being pro transparency.
I agree with all of these, though I think it's difficult to know how to weigh them against the things they sometimes trade-off against. For instance, if you're deciding whether to fund a large and potentially very impactful grant opportunity that involves high salaries, where the details of the grant are sensitive, it feels unclear to me how much impact there has to be on the table to justify the high salaries and discretion the grant involves. I'm pulled towards posts like this, though I likewise feel some pull towards the ideas Will discussed in the EA and the current funding situation (or at least my memory of it, which is that in some cases it's worth setting aside our preference for an ascetic aesthetic when resources are more plentiful and the world's problems are urgent.)
The thing I'm especially curious about is exactly how power and influence can start corrupting one's thinking, and what ways there are of avoiding that (to the extent there are any).
I started my EA journy at Giving What We Can, encouraging others to pledge to give 10% of their income to effective charities. I got into allt his because I thought it was basically insane that you could save a life for $5k. I still do, but now I'm working in AI, where there's significant amounts of funding available, powerful people involved and interested, the stakes are existential, I feel like my scope sensitivty cannot keep up anymore. So lot of what you're saying resonates.
I think EA’s failure to grapple with the corrupting influence of power is among its greatest failures.
I both agree with this in the abstract and also feel pretty clueless about what grappling with this would look like. I worry that one way to grapple with it is to try to avoid it and flee to a place with less moral ambiguity, but also at the cost impact. I'd be interested in your thoughts on what it would look like for people who are wielding that power to take seriously its corrupting influence.
Fair enough — I think I was trying to say something along the lines of "going through any specific example invites a lot of genuinely thorny and difficult questions about counterfactuality/sign of impact/attribution to EA" (and again many of these are hard to discuss on a public forum) but I think zooming out, you can see EAs fingerprints in various important places. I think this leads to an overall common-sense perspective that EA has helped improve the situation.
Also, I agree I pointed to work in the middle of the ToC chain, but that seems kind of reasonable to me given that AI is currently not that powerful and not really that scary. AI hasn't yet been capable of causing a disaster, so it's not really possible to have prevented one (yet).
On the specific example of Redwood Research is doing a lot of really valuable safety work. I think pioneering Control has been a fairly useful accomplishment, and I suspect if someone wanted to dig into the details, they'd find that it was fairly counterfactual.
I guess as you disclaimed might be the case up front, I don't think these are the strongest or most informed examples of EAs impact on AI safety.
In many of cases of such impact, one can quibble about many things:
Whether that impact was clearly positive, or whether it had some kind of indirectly negative harmful effect, most commonly via speeding up AI development. See Paul Christiano's reflections on the impact of Reinforcement Learning with Human Feedback as an example.
The counterfactuality and persistence of the impact — e.g., like you said for many of these, would this have happened (eventually) anyway?
How attributable that was to EA (and unfortunately in some cases, due to EA having a toxic brand in many places, it's actually best if it is not that attributable to EA).
And last "Does any of that matter? All of EAs impact — for better or worse — has been its influence on Anthropic."
Yet, I think taken as a whole, I think EA has punched above its weight in many ways with respect to making AI go well. It's led to:
More and better staffed AI safety/security institutes
More and better staffed third-party evaluations, auditing, and science (METR, AVERI)
Large amounts of field-building that encourages talented people to work on making AI go well (MATS, BlueDot, 80k)
A significant amount of policy advocacy and public communications about AI risk.
Probably other examples, too.
A lot of the effort to make this happened relied on EA motivated people willing to take lower paid or less glamorous jobs.[1] While some specific organizations' or research or policy wins or public communications would have happened otherwise, but some wouldn't, and even still, happening earlier is still better.
I started out in EA caring about global health, and my first EA job was as a Researcher at GWWC. Even after becoming pretty convinced by AI risk and longtermism, I was still fairly sympathetic to concerns like "AI Safety alienating people". For instance, I was pretty against 80,000 Hours becoming explicitly focused on longtermism, and also pretty skeptical / worried about its pivot last year into leaning even more into AI. Now, looking at just how fast AI progress is developing, how much there is still be done to make it go well, and how valuable (I think) EA has been to date, I think I got a lot of that wrong.
And of course, in some cases, they happened to get pretty well-paid jobs that ended up being fairly glamorous (even if they weren't in the beginning). I don't think that undermines the impact much. I don't really begrudge the quant finance folks who give >50% of their income to charities, even if they're still pretty rich at the end of the day.
First, OpenAI may be successful in turning a large profit based on continued marginal improvements in AI systems, so that their company continues getting much more valuable, far faster than 5% annual growth, meaning that the endowment would grow in value, that is, on net accumulating rather than distributing wealth. In this case, OpenAI’s equity will appreciate greatly; it would be irresponsible for the nonprofit not to try to spend large parts of that increased value.
In this scenario, wouldn't it be much better if the non-profit didn't spend its money now? By holding onto the money now, it'd have much more to give later. Put another way: imagine if the grantees receiving the money were asked "would you prefer $100 today or $10,000 in 6 years?" many would take the latter.
One frame that might make this argument more compelling is that if OAI ends up building AGI and ends up having astronomical value, then the foundation is sitting on humanity's endowment. Spending it down now before it's realized its value could be very costly.
Curious what you think of the following objection: AI character work has to grapple with the question of whether we want AI systems to be “obedient” or “ethical". Yet, it seems non-obvious which is better for the long-term future because training them to be ethical might make alignment risk worse, and training them to be obedient makes misuse and other risks worse. So if you're uncertain about which is better — which I think you probably should be — then on expectation, the impact of affecting AI character is reduced proportional to how uncertain you are. I expect there are some ways you can affect AI character work that doesn't have this "obedience vs ethical" structure, but I also suspect that the argument made in this report about the importance of such work apply less to these interventions.
The example you gave is about marginal cost-effectiveness (we spend "$1 on ads"). I agree that then, in this abstract/idealized case, you should spend the $1 on ads. I think all the uncertainty you would realistically have makes it less obvious, though.
But average cost-effectiveness would be more like, we spent $1,000,000 on an organization that did a bunch of different activities, and we think that led to $1,500,000 counterfactually going to charity. This seems good on average, but there's a further question of whether we should give another $1 to the organization. And I think that the 6x figure of the orignal post is referring to average cost-effectiveness ("our current grantees deliver an average adjusted return on donations of 6x across our effective giving portfolio"). This is at least conceptually coherent with the bar for the marginal $ being closer to 1x.
I think you might find the GWWC impact evals interesting, they go into an enormous amount of depth on all these issues.
I don't think it does. It's conceptually coherent for an organization to have a very high average cost-effectiveness while also having a marginal cost-effectiveness below 1x. For this reason, I don't think you should have a "bar" for average cost-effectiveness. (You might be making the point that if the average cost-effectiveness is above 1, then you are better off making the grant than burning the money, and so it clears a bar in that sense, but it's not clear it's worth making the grant vs making a potentially much smaller grant, and so it's not a helpful 'bar' in the sense the term is usually used.)
Sure, but these are hard to account for. I agree it's better to adjust the model when it's possible, but you'll still be left with a model that has a tonne of uncertainty.
Yep! I wasn't trying to suggest you shouldn't account for that.
This could still be (and I'd guess is?) referring to the past and expected future average cost-effectiveness.
I also think that it'd be pretty reasonable to have a bar higher than 1x. (I don't know what CG's bar actually is.) There are many contentious choices you make when coming up with a multiplier — e.g., how do you discount future donations, how do you discount donations to less cost-effective charities, do you adjust for the opportunity cost of the labor of the employees who could otherwise do impactful work or eartn to give, etc. There's also just a huge amount of uncertainty in various places, especially around counterfactuality. So given all that, I think it'z reasonable to just zoom out and think: hmm, this intervention looks great overall, but I think the multiplier model isn't robust enough to justify further support of organizations that it estimates only have a 1.1x multiplier.
(Flagging I work at CG, but not on this team!)
Curious what you think of the Giving What We Can impact evaluation? I helped write the first one, which claimed that over the 2020–2022 period it had an average multiplier of 30x. I think the more recent one claimed it had a 6x multiplier.
I think there's an important distinction between average and marginal effectiveness — the above claim seems true in the abstract for orgs that can move >$1 on the margin but not on average. And IMO it's much harder to estimate marginal cost-effectiveness, because it's forward looking, whereas historical average cost-effectiveness is a bit easier.
Thanks for the kind words :) and appreciate the concreteness of these suggestions.
I think many of these seem to have a common thread of: treating money as a valuable resource, viewing profligacy as costly (both financially and to the soul), seeing signals of dedication and altruism as especially valuable, and generally being pro transparency.
I agree with all of these, though I think it's difficult to know how to weigh them against the things they sometimes trade-off against. For instance, if you're deciding whether to fund a large and potentially very impactful grant opportunity that involves high salaries, where the details of the grant are sensitive, it feels unclear to me how much impact there has to be on the table to justify the high salaries and discretion the grant involves. I'm pulled towards posts like this, though I likewise feel some pull towards the ideas Will discussed in the EA and the current funding situation (or at least my memory of it, which is that in some cases it's worth setting aside our preference for an ascetic aesthetic when resources are more plentiful and the world's problems are urgent.)
The thing I'm especially curious about is exactly how power and influence can start corrupting one's thinking, and what ways there are of avoiding that (to the extent there are any).
Thanks for writing this.
I started my EA journy at Giving What We Can, encouraging others to pledge to give 10% of their income to effective charities. I got into allt his because I thought it was basically insane that you could save a life for $5k. I still do, but now I'm working in AI, where there's significant amounts of funding available, powerful people involved and interested, the stakes are existential, I feel like my scope sensitivty cannot keep up anymore. So lot of what you're saying resonates.
I both agree with this in the abstract and also feel pretty clueless about what grappling with this would look like. I worry that one way to grapple with it is to try to avoid it and flee to a place with less moral ambiguity, but also at the cost impact. I'd be interested in your thoughts on what it would look like for people who are wielding that power to take seriously its corrupting influence.
Fair enough — I think I was trying to say something along the lines of "going through any specific example invites a lot of genuinely thorny and difficult questions about counterfactuality/sign of impact/attribution to EA" (and again many of these are hard to discuss on a public forum) but I think zooming out, you can see EAs fingerprints in various important places. I think this leads to an overall common-sense perspective that EA has helped improve the situation.
Also, I agree I pointed to work in the middle of the ToC chain, but that seems kind of reasonable to me given that AI is currently not that powerful and not really that scary. AI hasn't yet been capable of causing a disaster, so it's not really possible to have prevented one (yet).
On the specific example of Redwood Research is doing a lot of really valuable safety work. I think pioneering Control has been a fairly useful accomplishment, and I suspect if someone wanted to dig into the details, they'd find that it was fairly counterfactual.
I guess as you disclaimed might be the case up front, I don't think these are the strongest or most informed examples of EAs impact on AI safety.
In many of cases of such impact, one can quibble about many things:
Yet, I think taken as a whole, I think EA has punched above its weight in many ways with respect to making AI go well. It's led to:
A lot of the effort to make this happened relied on EA motivated people willing to take lower paid or less glamorous jobs.[1] While some specific organizations' or research or policy wins or public communications would have happened otherwise, but some wouldn't, and even still, happening earlier is still better.
I started out in EA caring about global health, and my first EA job was as a Researcher at GWWC. Even after becoming pretty convinced by AI risk and longtermism, I was still fairly sympathetic to concerns like "AI Safety alienating people". For instance, I was pretty against 80,000 Hours becoming explicitly focused on longtermism, and also pretty skeptical / worried about its pivot last year into leaning even more into AI. Now, looking at just how fast AI progress is developing, how much there is still be done to make it go well, and how valuable (I think) EA has been to date, I think I got a lot of that wrong.
And of course, in some cases, they happened to get pretty well-paid jobs that ended up being fairly glamorous (even if they weren't in the beginning). I don't think that undermines the impact much. I don't really begrudge the quant finance folks who give >50% of their income to charities, even if they're still pretty rich at the end of the day.
In this scenario, wouldn't it be much better if the non-profit didn't spend its money now? By holding onto the money now, it'd have much more to give later. Put another way: imagine if the grantees receiving the money were asked "would you prefer $100 today or $10,000 in 6 years?" many would take the latter.
One frame that might make this argument more compelling is that if OAI ends up building AGI and ends up having astronomical value, then the foundation is sitting on humanity's endowment. Spending it down now before it's realized its value could be very costly.
Thanks for this post!
Curious what you think of the following objection: AI character work has to grapple with the question of whether we want AI systems to be “obedient” or “ethical". Yet, it seems non-obvious which is better for the long-term future because training them to be ethical might make alignment risk worse, and training them to be obedient makes misuse and other risks worse. So if you're uncertain about which is better — which I think you probably should be — then on expectation, the impact of affecting AI character is reduced proportional to how uncertain you are. I expect there are some ways you can affect AI character work that doesn't have this "obedience vs ethical" structure, but I also suspect that the argument made in this report about the importance of such work apply less to these interventions.