Comments
41
Curated and popular this week
This is the third in a sequence of posts taken from my recent report: Why Did Environmentalism Become Partisan?
Summary
Rising partisanship did not make environmentalism more popular or politically effective. Instead, it saw flat or falling overall public opinion, fewer major legislative achievements, and fluctuating executive actions.
Public Opinion...
This post presents the executive summary from Giving What We Can’s impact evaluation for 2025. At the end of this post we share links to more information, including the full report and...
Recent opportunities to take action
At what level of compute spending will AI Safety research be cut off from being considered effective altruism (if any)?
Of course, saving humanity from misaligned AI could be argued to be close to priceless. But how many experiments have a direct theory of change (ToC) of how it's going to mitigate existential risk? Perhaps a general one is fine at low compute ("it only costs $10 and 'control research' is generally thought to be a good research agenda").
But what about $5,000? What about $10,000? These numbers start to compare to or surpass what organizations like Giving What We Can receive from someone who donates for a whole year. It also starts to compete with saving a human life via programmes like those in GiveWell's top charities.
What about $20,000? $30,000? $50,000? Over what time frame are we comfortable spending that much money on compute and still considering that money well (effectively) spent? A year? A month? A single experiment? What kind of discovery is worth $50,000 in AIS research? Should we expect a clear ToC?
I'm very pro AI Safety, but I'm worried about some of the numbers I'm hearing for compute budgets being thrown around (compared to the information gained). I'm wondering - is anyone else is worried about a movement being (famously) concerned with cost effectiveness continuing on this path? Should we encourage more accountability?
Is there a reason why you are focusing on compute and not salaries? The example numbers you use are rather low compared to the yearly salary of a single AIS researcher.
Fair question, I guess some of the numbers I've been hearing can wipe out a (high) yearly salary well within a month (or days).
Perhaps one layer deeper I generally "back" money spent on someone working on AIS full time for a year and think there will probably be some good to come out of that. Although it may happen quickly, it seems that at least some level of thought goes into which positions are needed to fill before the job posting goes out.
However, on individual experiments level, I think the level of scrutiny is much lower/potentially nonexistent.
There seem to be plausible arguments for paying market rate to retain top talent (although you may disagree with them), but I don't really think there's an argument to spend huge sums on experiments without even double-checking if there's a way this can reduce x-risk.
Right now I would give very little marginal philanthropic money to compute-based experiments. AI companies already do a lot of those, and I don't expect them to work anyway. ML experiments are not addressing the fundamental barriers to solving AI misalignment. A core problem is that experiments can't deal with the sharp left turn.
(I would make an exception for CaML-style alignment-to-animals work, but that's not about AI safety as it's normally construed.)
It seems very unlikely to me that there are AI safety projects that are both worth doing but also not worth doing if they spent $10k on compute. The human capital cost is typically far in excess of that.