Paul Christiano has a notion of competitiveness, which seems relevant. Directions and desiderata for AI control seems to be the the place it's stated most clearly.
The following quote (emphasis in the original) is one of the reasons he gives for desiring competitiveness, and seems to be in the same ballpark as the reason you gave:
You can’t unilaterally use uncompetitive alignment techniques; we would need global coordination to avoid trouble. If we _don’t know how to build competitive benign AI, then users/designers of AI systems have to compr_omise efficiency in order to maintain reliable control over those systems. The most efficient systems will by default be built by whoever is willing to accept the largest risk of catastrophe (or perhaps by actors who consider unaligned AI a desirable outcome).
It may be possible to avert this kind of race to the bottom by effective coordination by e.g. enforcing regulations which mandate adequate investments in alignment or restrict what kinds of AI are deployed. Enforcing such controls domestically is already a huge headache. But internationally things are even worse: a country that handicapped its AI industry in order to proceed cautiously would face the risk of being overtaken by a less prudent competitor, and avoiding that race would require effective international coordination.
Ultimately society will be able and willing to pay some efficiency cost to reliably align AI with human interests. But the higher that cost, the harder the coordination problem that we will need to solve. I think the research community should be trying to make that coordination problem as easy as possible.
This seems like an important worry. I've updated the main post to state that I'm now unclear whether reports are good or bad (because it seems like most of the effect comes from how others' use the information in the reports, and it's unclear to me whether they will mostly improve or worsen their judgement).
I do think that (a) people will discount lottery winners at least a bit relative to donors of the same size and (b) it's good to introduce input on funding evaluation from someone with errors that are (relatively) uncorrelated with major funding bodies' errors.
That the use of the funds will be worse when writing a report is plausible. Do you also think that reports change others' giving either negligibly or negatively?
I guess it depends on the details of the returns to scale for donors. If there are returns to scale across the whole range of possible values of the donor lottery, as long as one person who would do lots of work/has good judgment joins the donor lottery, we should be excited about less conscientious people joining as well.
To be more concrete, imagine the amount of good you can do with a donation goes with the square of the donation. Let's suppose one person who will be a good donor joins the lottery with $1. Everyone else in the lottery will make a neutral donation if they win. The expected value of the lottery is (good person's chance of winning) * (total pool)² = (1/total pool) * (total pool)² = total pool.
Obviously that exact model is a bit contrived, but it points at why non-report-writing people still bring value in a lottery
Sorry, I didn't communicate what I meant well there.
It might be the case that DALYs somewhat faithfully track both (a) the impact of conditions on subjective wellbeing and (b) the impact of conditions on economic contribution, even if they're not explicitly intended to track (b). It might also be the case that efforts to extend DALYs to more faithfully track (a) for things that are worse than death would mean that they tracked (b) less well in those cases.
Then, it could be the case that it's better to stick with the current way of doing things.
I don't actually think the above is particularly likely (and yet less so after writing it out) and even in the case that it captures something correct for some moral frameworks, it probably looks different under others.
If I remember/understand correctly, the normal instruments fail to deliver useful answers for very bad conditions. For example, if you administer a survey asking how many years of healthy life the survey-taker thinks a year where they suffer X is worth, very bad situations generate incredibly broad answers.
Some people say those years are valueless (so just at 0), some say they have huge disvalue (so they'd rather die now than face one year with the condition and then the rest of their life in good health), and some say that it's close to the value of a year of healthy life (not so sure about this one; I think I remember someone saying it, but it was just in conversation).
future studies should calibrate SWB ratings against an explicit lower anchor of death (I really do not see any need to estimate values for states worse than dead)
I didn't read around enough to see if he offers a justification for it.
One small note for DALYs as they currently are: if the major impacts of impairment are essentially flow-through effects of economic value (so if I'm bed-bound my whole life, that's bad for me, but the economic slowdown has overall worse effects), then it may be plausible that DALYs shouldn't go below zero. Your post mentions the case of the child, where it seems to go below zero. But it could be a practical point that keeping the measure at zero results in better estimations of the scale of economic impact than it going below zero.
I have been feeling the siren song of agent-based models recently (I think it seems a natural move in a lot of cases, because we are actually modelling agents), but your criticisms of them reminded me that they often don't pay for their complexity in better predictions. It seems quite a general and useful point, and perhaps could be extracted to a standalone post, if you had the time and inclination.
I know it wasn't a major area of focus for you, but do you have a vague impression of when randomisation might be a big win purely by reducing costs of evaluation? One particular case where it might be useful is funders where disbursement is bottlenecked by evaluation capacity. Do you have any pointers for useful places to start research on the idea?
I split out the comments into areas of concern over on lesswrong. I think it would be a bit too noisy to duplicate that over here, but do feel free to bring up any issues!
Seems right, though I don't know to what extent Paul's view is representative of OpenAI's overall view.
Paul Christiano has a notion of competitiveness, which seems relevant. Directions and desiderata for AI control seems to be the the place it's stated most clearly.
The following quote (emphasis in the original) is one of the reasons he gives for desiring competitiveness, and seems to be in the same ballpark as the reason you gave:
This seems like an important worry. I've updated the main post to state that I'm now unclear whether reports are good or bad (because it seems like most of the effect comes from how others' use the information in the reports, and it's unclear to me whether they will mostly improve or worsen their judgement).
I do think that (a) people will discount lottery winners at least a bit relative to donors of the same size and (b) it's good to introduce input on funding evaluation from someone with errors that are (relatively) uncorrelated with major funding bodies' errors.
That the use of the funds will be worse when writing a report is plausible. Do you also think that reports change others' giving either negligibly or negatively?
I guess it depends on the details of the returns to scale for donors. If there are returns to scale across the whole range of possible values of the donor lottery, as long as one person who would do lots of work/has good judgment joins the donor lottery, we should be excited about less conscientious people joining as well.
To be more concrete, imagine the amount of good you can do with a donation goes with the square of the donation. Let's suppose one person who will be a good donor joins the lottery with $1. Everyone else in the lottery will make a neutral donation if they win. The expected value of the lottery is (good person's chance of winning) * (total pool)² = (1/total pool) * (total pool)² = total pool.
Obviously that exact model is a bit contrived, but it points at why non-report-writing people still bring value in a lottery
Thanks for highlighting in this comment. It don't think I made that prominent enough in the post itself
Sorry, I didn't communicate what I meant well there.
It might be the case that DALYs somewhat faithfully track both (a) the impact of conditions on subjective wellbeing and (b) the impact of conditions on economic contribution, even if they're not explicitly intended to track (b). It might also be the case that efforts to extend DALYs to more faithfully track (a) for things that are worse than death would mean that they tracked (b) less well in those cases.
Then, it could be the case that it's better to stick with the current way of doing things.
I don't actually think the above is particularly likely (and yet less so after writing it out) and even in the case that it captures something correct for some moral frameworks, it probably looks different under others.
I agree that this seems important.
If I remember/understand correctly, the normal instruments fail to deliver useful answers for very bad conditions. For example, if you administer a survey asking how many years of healthy life the survey-taker thinks a year where they suffer X is worth, very bad situations generate incredibly broad answers.
Some people say those years are valueless (so just at 0), some say they have huge disvalue (so they'd rather die now than face one year with the condition and then the rest of their life in good health), and some say that it's close to the value of a year of healthy life (not so sure about this one; I think I remember someone saying it, but it was just in conversation).
As far as discussion on it, I found this GiveWell post that glances on it, and health economist Paul Dolan saying
I didn't read around enough to see if he offers a justification for it.
One small note for DALYs as they currently are: if the major impacts of impairment are essentially flow-through effects of economic value (so if I'm bed-bound my whole life, that's bad for me, but the economic slowdown has overall worse effects), then it may be plausible that DALYs shouldn't go below zero. Your post mentions the case of the child, where it seems to go below zero. But it could be a practical point that keeping the measure at zero results in better estimations of the scale of economic impact than it going below zero.
on the greaterwrong version of the EA forum, there's an automatically generated TOC. So that's an option for people who would strongly prefer a TOC
I have been feeling the siren song of agent-based models recently (I think it seems a natural move in a lot of cases, because we are actually modelling agents), but your criticisms of them reminded me that they often don't pay for their complexity in better predictions. It seems quite a general and useful point, and perhaps could be extracted to a standalone post, if you had the time and inclination.
I know it wasn't a major area of focus for you, but do you have a vague impression of when randomisation might be a big win purely by reducing costs of evaluation? One particular case where it might be useful is funders where disbursement is bottlenecked by evaluation capacity. Do you have any pointers for useful places to start research on the idea?
I split out the comments into areas of concern over on lesswrong. I think it would be a bit too noisy to duplicate that over here, but do feel free to bring up any issues!