Comments
Medium term AI forecasting with Metaculus
I'm working on a collection of metaculus.com questions intended to generate AI domain specific forecasting insights. These questions are intended to resolve in the 1-15 year range, and my hope is that if they're sufficiently independent, we'll get a range of positive and negative resolutions which will inform future forecasts.
I've already gotten a couple of them live, and am hoping for feedback on the rest:
1. When will AI out-perform humans on argument reasoning tasks?
2. When will multi-modal ML out-perform uni-modal ML?
3. (Not by me) When will image recognition be made robust against unrestricted adversary?
4. (WIP) When will reinforcement learning methods achieve sample efficiency within four orders of magnitude of human efficiency?*
5. (WIP) When will unsupervised learning methods achieve human level performance on image classification?
The more questions the better, so please make suggestions. Of course we have to avoid burdening the good folks working at metaculus, so 8-10 questions is probably the maximum I'd be willing to personally submit.
*I am not very familiar with reinforcement learning, so input here would be particularly helpful! What is the best way to operationalize this question? How many orders of magnitude? Is there a relevant benchmark? etc. I'd be happy for someone else to take the credit, and post the question themselves as well!
QALY/$ for promoting zinc as a common cold intervention
Epistemic status: Fun speculation. I know nothing about public health, and grabbed numbers from the first source I could find for every step of the below. I link to the sources which informed my point estimates.
Here’s my calculation broken down into steps:
Health-related quality of life effect for one year of common cold -0.2
Common cold prevalence in the USA 1.2/yr
Modally 7 days of symptoms having -0.2
~1.5 million QALY burden per year when aggregated across the US population
50% probability the right zinc lozenges with proper dosing can prevent >90% of colds. This comes from here, here, and my personal experience of taking zinc lozenges ~10ish occasions.
15% best case adoption scenario, from taking a log-space mean of
100,000 QALYs/year is my estimate for the expected value of taking some all-or-nothing action to promote zinc lozenges (without the possibility of cheaply confirming whether they work) which successfully changes public knowledge and medical advice to promote our best-guess protocol for taking zinc.
$35 million is my estimate for how much we should be willing to spend to remain competitive with Givewell’s roughly 1 QALY/$71. This assumes a 5 year effect duration. I have no idea how much such a thing would cost but I’d guess at most 1 OOM of value is being left on the table here, so I’m a bit less bullish on Zinc than I was before calculating.
EDIT: I calculated the cost of supplying the lozenges themselves. Going off these price per lozenge, this 5 year USA supply of lozenges costs ~35 million alone. Presumably this doesn't need to hit the Givewell spending bar, but just US government spending on healthcare.
It feels like this is still a research problem needing larger scale trials. If the claims are true (i.e. the failures to achieve statistically significant results were due to not preparing and consuming lozenges in a particular way, rather than the successes being the anomalies) there are plenty of non-philanthropic entities (governments and employers and media as well as zinc supplement vendors) that would be incentivised to publicise more widely.
Is there a meta-analysis studying the effect size of this intervention? These seem unrealistically high to me.
Lifeextension cites this https://pubmed.ncbi.nlm.nih.gov/24715076/ claiming "The results showed that when the proper dose of zinc is used within 24 hours of first symptoms, the duration of cold miseries is cut by about 50%" I'd be interested if you do a dig through the citation chain. The lifeextension page has a number of further links.
That citation is retracted?
Here's the Cochrane withdrawal notice.
Looks like they plagiarized from this paper, which found:
Good catch, thanks.
I'm seeing 0.25% globally and 0.31% for the US for URI in the GBD data, ~1 OOM lower (the direct figure for the US is 3.4e5, also ~1 OOM lower). What am I missing?