"we haven't had examples where a huge amount of cognitive labour has been dumped on a scientific field and we've been able to observe how much progress in that field accelerates"
Well, Claude 3.5 and I can think of some examples that contradict that statement. These are Claude's estimates:
The rise of citizen science and crowdsourcing in certain fields. For instance, projects like Galaxy Zoo in astronomy have allowed large numbers of amateur scientists to contribute to data analysis, accelerating progress in classifying galaxies.
Duration: Ongoing since 2007 (about 17 years)
Degree of influx: Substantial. Galaxy Zoo alone has involved over 250,000 volunteers.
Acceleration: Significant but focused. The original Galaxy Zoo project classified ~900,000 galaxies in less than a year, a task that would have taken years for professional astronomers. However, the acceleration is primarily in data processing rather than theoretical advancements.
Estimated acceleration: 10-20x faster for specific classification tasks, but perhaps only 2-3x acceleration for the field of galaxy morphology as a whole.
The influx of physicists and mathematicians into quantitative finance and economics in the 1980s and 1990s. This led to rapid developments in financial modeling and econometrics.
Duration: About 20 years (concentrated influx)
Degree of influx: Moderate. Estimated several thousand PhDs over this period.
Acceleration: Substantial. This influx led to the rapid development of complex financial models and the growth of quantitative trading.
Estimated acceleration: 5-10x in areas like options pricing and risk modeling. Overall acceleration in quantitative finance might be around 3-5x.
The growth of computer science departments in universities during the 1960s and 1970s, which led to an acceleration in theoretical computer science and algorithm development.
Duration: About 20 years
Degree of influx: Significant. The number of CS departments and graduates grew rapidly during this period.
Acceleration: Major. This period saw fundamental developments in algorithms, programming languages, and theoretical computer science.
Estimated acceleration: 5-8x in theoretical computer science and algorithm development. The overall field might have seen a 3-4x acceleration.
I think it's also interesting to see how open source contributions to language models and academia clearly havr thousands of times more contributors but seems to make relatively limited progress compared to the top AI labs. The main reason being, presumably, the lack of compute for experiments and training. So that's one reason to be less concerned about a major influx of cognitive skills with limited compute.
Agreed. Carl Schuman at hour 1:02 at the 80k podcast even notes: https://80000hours.org/podcast/episodes/carl-shulman-common-sense-case-existential-risks/ Rob Wiblin: I see. So because there’s such a clear motivation for even an altruistic person to exaggerate the potential risk from nuclear winter, then people who haven’t looked into it might regard the work as not super credible because it could kind of be a tool for advocacy more than anything.
Carl Shulman: Yeah. And there was some concern of that sort, that people like Carl Sagan, who was both an anti-nuclear and antiwar activist and bringing these things up. So some people, particularly in the military establishment, might have more doubt about when their various choices in the statistical analysis and the projections and assumptions going into the models, are they biased in this way? And so for that reason, I’ve recommended and been supportive of funding, just work to elaborate on this. But then I have additionally especially valued critical work and support for things that would reveal this was wrong if it were, because establishing that kind of credibility seemed very important. And we were talking earlier about how salience and robustness and it being clear in the minds of policymakers and the public is important.
Note earlier in the conversation demonstrating Schulman influenced the funding decision for the Rutgers team from open philanthropy: "Robert Wiblin: So, a couple years ago you worked at the Gates Foundation and then moved to the kind of GiveWell/Open Phil cluster that you’re helping now."
Notably, Reisner is part of Los Alamos in the military establishment. They build nuclear weapons there. So both Reisner and Robock from Rutgers have their own biases.
It seems like a much less biased middle ground, and generally shows that nuclear winter is still really bad, on the order of 1/2 to 1/3 as "bad" as Rutgers tends to say it is.
As a disclaimer, I came in with the preconception that one should assign near-zero probability of animals being of more moral relevance than humans.
After reading the arguments, I have found little to no convincing arguments contradicting this.
It's true that we should be uncertain as to how animals experience the world. However, I don't feel that the uncertainty in moral value should be thought of as ever exceeding human's moral value.
To illustrate my current understanding of the best way to think about this topic, I think all your probability distributions should probably be modeled as never exceeding 1 for every animal, as the probability of such an outcome is so low it's not worth considering. I think of it like the probability that you can build a perpetual energy-creating machine violating the laws of physics, or the probability that tomorrow the sun does not rise because the earth stopped rotating.
Perhaps, it could analogized as the same moral probability that causing suffering is a good thing, all things considered. One might argue that the human brain is extremely complicated, and morality is complicated, so we should put some weight on moral views that prefer to cause infinite suffering for eternity. Perhaps one could argue that some people enjoy causing others to suffer, and they might be right, and so suffering might be intrinsically good. I think this argument has about as much supporting evidence as the concept that animals could be more morally relevant than people. However, again, I would say the probability of such an outcome is so low it's not worth considering.
Although it's true we do not know the details of how animals experience consciousness, this is not enough to overturn the intuition all humans share about the morality of killing people versus animals -- one is simply entirely different than another, and there is no instance in which it is better to kill an animal than a person. This conception has apparently been held constant for many cultures throughout human history. In some cases some animals were revered as gods, but this was less about the animals and more about the gods. In some cases animals and living things were seen as equally valuable as humans. I think this is unlikely, but not impossible, but the key point is that killing was seen as wrong in all cases, and not that animals were seen as more valuable than humans.
Suffering is not the only relevant moral consideration. See "The Righteous Mind" by Jonathon Haidt -- humans probably share a few more moral foundations than purely care/harm, including authority, fairness, sanctity, etc. Some may view these as equally morally relevant. My point is here, it's questionable whether we have equal moral responsibility over nonhuman animals as we have to humans, depending on how you construct your moral frameworks. If you look at how human brains are wired, the foundations of our conceptions of morality are built with in-group vs out-group. So, the moral status of animals based on understanding of human psychology which is our best way to guess at a "correct" moral framework would indicate that as things become less like us, our moral intuitions will guide us as valuing these things less.
I think you may have come to your probability distributions because you are a sequence thinker and are using your intuitions to argue for each part of a sequence which comes to some conclusion, where the proper thing to do when coming to some conclusion about whether to spend on an animal welfare charity or not is to use cluster-style thinking.
I hope that this is seen as a respectful difference in perspective and not at all a personal attack. I think it is useful to question these sort of assumptions to make moral progress, but I also think we need a lot of evidence to overturn the assumption that humans are more or equally morally relevant than animals, in large part due to the pre-existing moral intuitions we all probably share. There don't appear to be sufficient arguments out there to overturn this position.
Okay, that was enough philosophizing, let me put in a few more points in favor of my position here:
Most people I know that are smarter than me believe humans are more morally significantly than animal. I know of zero people seriously arguing the opposite side
If morality is actually all fake and a human invention with no objective truth to it, then humans and animals will both be worth zero, and I will still be correct.
The actual actions of people who argue animals are more morally relevant than humans is not to kill people to save animals, so there's probably no-one who sincerely, deep down believes this
People tend to anthropomorphize other things like teddy bears and Roombas and things like that, and mistakenly assign them some moral worth until they think about it more. Therefore, our intuitions can tend to guide us to incorrect conclusions about what is morally worthwhile.
To pick a bit on the notion from this article which establishes the range of moral weights in question.
They say fruit flies range from a moral weight of .000001 - 20 times the moral significance of human experience. In log space, that's between 10^-6 and 10^1.3. The mean log uniform distribution, as you mention, is at 1.95. I find the significant probability mass being above 1 as implausible for fruit flies, and I will go on to explain why I think except for species like dogs, pigs, elephants, octopuses, or other long living intelligent social creatures it would be difficult to argue that they are plausibly more moral weight than a human.
Arguments for fruit flies being about as likely to be more morally significant than humans as less:
A fruit fly may experience things much faster than a human, meaning their short lives may be experienced longer than how long a human may perceive their lives to be.
They may experience things more intensely as well, given their single pointed focus on conscious experience. This means although it's possible they experience suffering more fully, it may also be possible that they can completely forget and move their focus to some other task if focusing on the pain does not confer an advantage.
They may be less "distracted" than humans, in that they experience the world more fully and in full awareness.
They are also typically considered innocent of other moral wrongdoings, so perhaps that makes them more morally valuable in some moral systems.
Arguments against:
There are a whole host of reasons to think that they could not possibly be as morally significant as a human.
I think it's reasonable to say a fruit fly cannot remember things in the long term, and it cannot contemplate or ruminate, which is one of the worst aspects of negative experiences and pain. I think most people would prefer to have experiences of extreme pain and trauma erased from their lives.
A fruit fly lives a tiny fraction of the duration of a human's life, so it would have to experience its own life much faster.
A human can be considered an ensemble or family of different personalities and conscious processes. Each one of these may have moral significance, increasing the relative moral significance of a human.
The more complex something is, typically, it is more valued in generic terms.
Humans form a network of social connections and social connections. When a human is lost, their loss is understood and grieved by many other humans, thus greatly increasing the overall negative effect of harm to a human compared to a fruit fly.
Humans have very few children relative to fruit fly, so they are likely value higher on an individual level by their families and communities.
In summary the most relevant factors for moral significance are likely the degree of social embeddedness, the experience of higher order emotions and complexity in general, the ability to grieve, long lives, and long memories, which strongly implies that humans are more morally significant than all or most other animals.
A final thought is that we don't know with very high confidence that animals are conscious in the way that we care about morally, but we know this for sure with humans. For that reason, we would be safer to prefer to save humans first, in case we were wrong about animals having conscious experiences in the first place.
"we haven't had examples where a huge amount of cognitive labour has been dumped on a scientific field and we've been able to observe how much progress in that field accelerates"
Well, Claude 3.5 and I can think of some examples that contradict that statement. These are Claude's estimates:
The rise of citizen science and crowdsourcing in certain fields. For instance, projects like Galaxy Zoo in astronomy have allowed large numbers of amateur scientists to contribute to data analysis, accelerating progress in classifying galaxies. Duration: Ongoing since 2007 (about 17 years) Degree of influx: Substantial. Galaxy Zoo alone has involved over 250,000 volunteers. Acceleration: Significant but focused. The original Galaxy Zoo project classified ~900,000 galaxies in less than a year, a task that would have taken years for professional astronomers. However, the acceleration is primarily in data processing rather than theoretical advancements. Estimated acceleration: 10-20x faster for specific classification tasks, but perhaps only 2-3x acceleration for the field of galaxy morphology as a whole.
The influx of physicists and mathematicians into quantitative finance and economics in the 1980s and 1990s. This led to rapid developments in financial modeling and econometrics. Duration: About 20 years (concentrated influx) Degree of influx: Moderate. Estimated several thousand PhDs over this period. Acceleration: Substantial. This influx led to the rapid development of complex financial models and the growth of quantitative trading. Estimated acceleration: 5-10x in areas like options pricing and risk modeling. Overall acceleration in quantitative finance might be around 3-5x.
The growth of computer science departments in universities during the 1960s and 1970s, which led to an acceleration in theoretical computer science and algorithm development. Duration: About 20 years Degree of influx: Significant. The number of CS departments and graduates grew rapidly during this period. Acceleration: Major. This period saw fundamental developments in algorithms, programming languages, and theoretical computer science. Estimated acceleration: 5-8x in theoretical computer science and algorithm development. The overall field might have seen a 3-4x acceleration.
I think it's also interesting to see how open source contributions to language models and academia clearly havr thousands of times more contributors but seems to make relatively limited progress compared to the top AI labs. The main reason being, presumably, the lack of compute for experiments and training. So that's one reason to be less concerned about a major influx of cognitive skills with limited compute.
Agreed. Carl Schuman at hour 1:02 at the 80k podcast even notes:
https://80000hours.org/podcast/episodes/carl-shulman-common-sense-case-existential-risks/
Rob Wiblin: I see. So because there’s such a clear motivation for even an altruistic person to exaggerate the potential risk from nuclear winter, then people who haven’t looked into it might regard the work as not super credible because it could kind of be a tool for advocacy more than anything.
Carl Shulman: Yeah. And there was some concern of that sort, that people like Carl Sagan, who was both an anti-nuclear and antiwar activist and bringing these things up. So some people, particularly in the military establishment, might have more doubt about when their various choices in the statistical analysis and the projections and assumptions going into the models, are they biased in this way? And so for that reason, I’ve recommended and been supportive of funding, just work to elaborate on this. But then I have additionally especially valued critical work and support for things that would reveal this was wrong if it were, because establishing that kind of credibility seemed very important. And we were talking earlier about how salience and robustness and it being clear in the minds of policymakers and the public is important.
Note earlier in the conversation demonstrating Schulman influenced the funding decision for the Rutgers team from open philanthropy:
"Robert Wiblin: So, a couple years ago you worked at the Gates Foundation and then moved to the kind of GiveWell/Open Phil cluster that you’re helping now."
Notably, Reisner is part of Los Alamos in the military establishment. They build nuclear weapons there. So both Reisner and Robock from Rutgers have their own biases.
Here's a peer-reviewed perspective that shows the flaws in both perspectives on nuclear winter as being too extreme:
https://www.tandfonline.com/doi/pdf/10.1080/25751654.2021.1882772
I recommend Lawrence livermore paper on the topic: https://www.osti.gov/biblio/1764313
It seems like a much less biased middle ground, and generally shows that nuclear winter is still really bad, on the order of 1/2 to 1/3 as "bad" as Rutgers tends to say it is.
As a disclaimer, I came in with the preconception that one should assign near-zero probability of animals being of more moral relevance than humans.
After reading the arguments, I have found little to no convincing arguments contradicting this.
It's true that we should be uncertain as to how animals experience the world. However, I don't feel that the uncertainty in moral value should be thought of as ever exceeding human's moral value.
To illustrate my current understanding of the best way to think about this topic, I think all your probability distributions should probably be modeled as never exceeding 1 for every animal, as the probability of such an outcome is so low it's not worth considering. I think of it like the probability that you can build a perpetual energy-creating machine violating the laws of physics, or the probability that tomorrow the sun does not rise because the earth stopped rotating.
Perhaps, it could analogized as the same moral probability that causing suffering is a good thing, all things considered. One might argue that the human brain is extremely complicated, and morality is complicated, so we should put some weight on moral views that prefer to cause infinite suffering for eternity. Perhaps one could argue that some people enjoy causing others to suffer, and they might be right, and so suffering might be intrinsically good. I think this argument has about as much supporting evidence as the concept that animals could be more morally relevant than people. However, again, I would say the probability of such an outcome is so low it's not worth considering.
Although it's true we do not know the details of how animals experience consciousness, this is not enough to overturn the intuition all humans share about the morality of killing people versus animals -- one is simply entirely different than another, and there is no instance in which it is better to kill an animal than a person. This conception has apparently been held constant for many cultures throughout human history. In some cases some animals were revered as gods, but this was less about the animals and more about the gods. In some cases animals and living things were seen as equally valuable as humans. I think this is unlikely, but not impossible, but the key point is that killing was seen as wrong in all cases, and not that animals were seen as more valuable than humans.
Suffering is not the only relevant moral consideration. See "The Righteous Mind" by Jonathon Haidt -- humans probably share a few more moral foundations than purely care/harm, including authority, fairness, sanctity, etc. Some may view these as equally morally relevant. My point is here, it's questionable whether we have equal moral responsibility over nonhuman animals as we have to humans, depending on how you construct your moral frameworks. If you look at how human brains are wired, the foundations of our conceptions of morality are built with in-group vs out-group. So, the moral status of animals based on understanding of human psychology which is our best way to guess at a "correct" moral framework would indicate that as things become less like us, our moral intuitions will guide us as valuing these things less.
I think you may have come to your probability distributions because you are a sequence thinker and are using your intuitions to argue for each part of a sequence which comes to some conclusion, where the proper thing to do when coming to some conclusion about whether to spend on an animal welfare charity or not is to use cluster-style thinking.
I hope that this is seen as a respectful difference in perspective and not at all a personal attack. I think it is useful to question these sort of assumptions to make moral progress, but I also think we need a lot of evidence to overturn the assumption that humans are more or equally morally relevant than animals, in large part due to the pre-existing moral intuitions we all probably share. There don't appear to be sufficient arguments out there to overturn this position.
Okay, that was enough philosophizing, let me put in a few more points in favor of my position here:
To pick a bit on the notion from this article which establishes the range of moral weights in question.
They say fruit flies range from a moral weight of .000001 - 20 times the moral significance of human experience. In log space, that's between 10^-6 and 10^1.3. The mean log uniform distribution, as you mention, is at 1.95. I find the significant probability mass being above 1 as implausible for fruit flies, and I will go on to explain why I think except for species like dogs, pigs, elephants, octopuses, or other long living intelligent social creatures it would be difficult to argue that they are plausibly more moral weight than a human.
Arguments for fruit flies being about as likely to be more morally significant than humans as less:
A fruit fly may experience things much faster than a human, meaning their short lives may be experienced longer than how long a human may perceive their lives to be.
They may experience things more intensely as well, given their single pointed focus on conscious experience. This means although it's possible they experience suffering more fully, it may also be possible that they can completely forget and move their focus to some other task if focusing on the pain does not confer an advantage.
They may be less "distracted" than humans, in that they experience the world more fully and in full awareness.
They are also typically considered innocent of other moral wrongdoings, so perhaps that makes them more morally valuable in some moral systems.
Arguments against:
There are a whole host of reasons to think that they could not possibly be as morally significant as a human.
I think it's reasonable to say a fruit fly cannot remember things in the long term, and it cannot contemplate or ruminate, which is one of the worst aspects of negative experiences and pain. I think most people would prefer to have experiences of extreme pain and trauma erased from their lives.
A fruit fly lives a tiny fraction of the duration of a human's life, so it would have to experience its own life much faster.
A human can be considered an ensemble or family of different personalities and conscious processes. Each one of these may have moral significance, increasing the relative moral significance of a human.
The more complex something is, typically, it is more valued in generic terms.
Humans form a network of social connections and social connections. When a human is lost, their loss is understood and grieved by many other humans, thus greatly increasing the overall negative effect of harm to a human compared to a fruit fly.
Humans have very few children relative to fruit fly, so they are likely value higher on an individual level by their families and communities.
In summary the most relevant factors for moral significance are likely the degree of social embeddedness, the experience of higher order emotions and complexity in general, the ability to grieve, long lives, and long memories, which strongly implies that humans are more morally significant than all or most other animals.
A final thought is that we don't know with very high confidence that animals are conscious in the way that we care about morally, but we know this for sure with humans. For that reason, we would be safer to prefer to save humans first, in case we were wrong about animals having conscious experiences in the first place.