A preliminary estimate, and a request for better ones.
Summary
I believe the standard literature estimates for the number of DALYs attributable to a case of stunting are too low, largely because they don’t account for the long term effects. This means that childhood nutritional interventions that reduce the prevalence of stunting may be substantially more cost-effective than previously believed.
Epistemic status
Exploratory and back-o...
Note: This post was crossposted from Planned Obsolescence by the Forum team, with the author's permission. The author may not see or respond to comments on this post.
Subtitle: It’s a major warning shot, and might be the last one we get
All opinions are my personal view, and don’t represent my employer or fellow investigators.
This week, METR and Redwood Research published...
Last week, the Forum grappled with cluelessness. The volume of entries (76 essays!)[1]—from newcomers...
At what level of compute spending will AI Safety research be cut off from being considered effective altruism (if any)?
Of course, saving humanity from misaligned AI could be argued to be close to priceless. But how many experiments have a direct theory of change (ToC) of how it's going to mitigate existential risk? Perhaps a general one is fine at low compute ("it only costs $10 and 'control research' is generally thought to be a good research agenda").
But what about $5,000? What about $10,000? These numbers start to compare to or surpass what organizations like Giving What We Can receive from someone who donates for a whole year. It also starts to compete with saving a human life via programmes like those in GiveWell's top charities.
What about $20,000? $30,000? $50,000? Over what time frame are we comfortable spending that much money on compute and still considering that money well (effectively) spent? A year? A month? A single experiment? What kind of discovery is worth $50,000 in AIS research? Should we expect a clear ToC?
I'm very pro AI Safety, but I'm worried about some of the numbers I'm hearing for compute budgets being thrown around (compared to the information gained). I'm wondering - is anyone else is worried about a movement being (famously) concerned with cost effectiveness continuing on this path? Should we encourage more accountability?
Is there a reason why you are focusing on compute and not salaries? The example numbers you use are rather low compared to the yearly salary of a single AIS researcher.
Fair question, I guess some of the numbers I've been hearing can wipe out a (high) yearly salary well within a month (or days).
Perhaps one layer deeper I generally "back" money spent on someone working on AIS full time for a year and think there will probably be some good to come out of that. Although it may happen quickly, it seems that at least some level of thought goes into which positions are needed to fill before the job posting goes out.
However, on individual experiments level, I think the level of scrutiny is much lower/potentially nonexistent.
There seem to be plausible arguments for paying market rate to retain top talent (although you may disagree with them), but I don't really think there's an argument to spend huge sums on experiments without even double-checking if there's a way this can reduce x-risk.
Right now I would give very little marginal philanthropic money to compute-based experiments. AI companies already do a lot of those, and I don't expect them to work anyway. ML experiments are not addressing the fundamental barriers to solving AI misalignment. A core problem is that experiments can't deal with the sharp left turn.
(I would make an exception for CaML-style alignment-to-animals work, but that's not about AI safety as it's normally construed.)
It seems very unlikely to me that there are AI safety projects that are both worth doing but also not worth doing if they spent $10k on compute. The human capital cost is typically far in excess of that.