Thanks, very thought provoking and seems like a pleasant framework .
To help myself digest the implications let me try to distill the statements. I will intentionally make them slightly "stronger" than author's. I then see if I agree with those.
1. Generalizability is a smooth monotonic function of training data. 2.Current models are not being optimized for generalizability. 3. Training data are bandwidth limited by deployment.
The statement #1 is challenged by grokking. The step-like improvement in generalizability is expected. The question is where is the threshold. The statement #2 seems to have been true up to date. I am aware of at least one startup case https://www.xentlabs.ai/ that recognized this insight and now explicitly optimizes for generalizability. The statement #3 will hold if a) synthetic data are not sufficient for grokking generalizability b) there is no easy-for-deployment industries, that will provide sufficient training data at a low cost. Is SaaS such an industry?
After all this I tend to believe that the road-block is not particularly real. The deployment, partial and targeted on easy industries, will continue to happen and generalizability will grow. In the business context we will have something akin to RLHF but RLManagerF. The "grokking threshold," is naturally unknown.
"implying power dependence on GDP" means that the quantity on the Y-axis is the power function of GDP, i.e. GDP^x. It looks like Maternal deaths ~ GDP^(-2), that is for every order of magnitude increase in GDP we have two orders of magnitude decrease in maternal mortality. This is very different from the logarithmic dependence one obtains for straight lines in log-linear plots.
Thanks for the piece. It brings me two contradicting emotions of warmth and unsettling sadness for "excluding" the rest of humanity. I don't think I need to explain the first one, but I want to explore the second.
I note two things which I can unite under the "honest EA" concept: 1) EA, as a concept, is a very human way of thinking. If you ask your average Joe if they want to do good, they'd likely say "yes," and if you'd ask them if they'd like to do it effectively, they'd probably also say "yes." So, I really believe that an honest version of EA is close to universal moral (*might be too universal, honestly, but that is a problem with the word "good") 2) All three features you point out, as well as point 1) above, are not binary. For each individual, there is a distribution of the range of empathy, there is scope sensitivity (it is just that it gets overridden by moral circle concerns), and surely there are environments and conditions in which almost any human can experience the scout mindset, just that few people bother to create those environments. Being effective in one's altruism is also the point on the distribution.
As you notice, we appreciate that it's ok to care more about our family and friends, and in those moments, we are not "absolute EA," but we are very normal humans.
I believe that honestly appreciating the fact that "we are just points on the distribution that, due to a privilege of economic, intellectual or emotional stability, are on the "high" end of the distribution" can provide us with the humility to empathize with a fellow non-EA human being and recognize that the three features you've listed, are, in fact, everywhere. It is just that it takes much more than these three qualities to make a human.
I believe this recognition is essential for the future of the community and the psychological health of the citizens of this forum. I want to talk to non-EA, not as to someone who doesn't share my values, but to someone who wasn't lucky enough to have a chance to make space for EA activities in their days, minds, and hearts but deep inside, we share the same honest EA idea "we aim to do good effectively, while also doing those other mysterious things that make us into a wholesome human being."
*This might be the very same way you feel. But I still thought it was important to share.
Great article. I would suggest correcting the sentence "On the horizontal axis in each panel you see GDP per capita..." as what we see is a logarithm of the GDP per capita. While this doesn't challenge any of the messages Max is making, one will get a very distorted perception if one assumes that all those metrics are linearly correlated with the GDPpc instead of the logarithm of the GDPpc. In practice, if we consider increasing GDPpc by 1000USD on the left side of the graphs it will shift the point by 30% of the whole range, while if we consider the reduction of the rightmost points by 1000 USD the shift will be ~1%.
This misreading happened to me on the first read and I have also heard other people saying that various metrics are correlated with the GDP, referring to this article.
Another even more nitpicky comment: Although I understand that Max is simply reusing graphics from an external source, I should note that one might be deceived by the presentation in two other cases: - in graphs 2 and 3 as the vertical axis is also logarithmic (implying power dependence on GDP). - in the first and the last two graphs' Y-axis doesn't start from zero
Thanks, very thought provoking and seems like a pleasant framework .
To help myself digest the implications let me try to distill the statements. I will intentionally make them slightly "stronger" than author's. I then see if I agree with those.
1. Generalizability is a smooth monotonic function of training data.
2.Current models are not being optimized for generalizability.
3. Training data are bandwidth limited by deployment.
The statement #1 is challenged by grokking. The step-like improvement in generalizability is expected. The question is where is the threshold.
The statement #2 seems to have been true up to date. I am aware of at least one startup case https://www.xentlabs.ai/ that recognized this insight and now explicitly optimizes for generalizability.
The statement #3 will hold if
a) synthetic data are not sufficient for grokking generalizability
b) there is no easy-for-deployment industries, that will provide sufficient training data at a low cost. Is SaaS such an industry?
After all this I tend to believe that the road-block is not particularly real. The deployment, partial and targeted on easy industries, will continue to happen and generalizability will grow. In the business context we will have something akin to RLHF but RLManagerF.
The "grokking threshold," is naturally unknown.
"implying power dependence on GDP" means that the quantity on the Y-axis is the power function of GDP, i.e. GDP^x. It looks like Maternal deaths ~ GDP^(-2), that is for every order of magnitude increase in GDP we have two orders of magnitude decrease in maternal mortality.
This is very different from the logarithmic dependence one obtains for straight lines in log-linear plots.
Thanks for the piece. It brings me two contradicting emotions of warmth and unsettling sadness for "excluding" the rest of humanity. I don't think I need to explain the first one, but I want to explore the second.
I note two things which I can unite under the "honest EA" concept:
1) EA, as a concept, is a very human way of thinking. If you ask your average Joe if they want to do good, they'd likely say "yes," and if you'd ask them if they'd like to do it effectively, they'd probably also say "yes." So, I really believe that an honest version of EA is close to universal moral (*might be too universal, honestly, but that is a problem with the word "good")
2) All three features you point out, as well as point 1) above, are not binary. For each individual, there is a distribution of the range of empathy, there is scope sensitivity (it is just that it gets overridden by moral circle concerns), and surely there are environments and conditions in which almost any human can experience the scout mindset, just that few people bother to create those environments. Being effective in one's altruism is also the point on the distribution.
As you notice, we appreciate that it's ok to care more about our family and friends, and in those moments, we are not "absolute EA," but we are very normal humans.
I believe that honestly appreciating the fact that "we are just points on the distribution that, due to a privilege of economic, intellectual or emotional stability, are on the "high" end of the distribution" can provide us with the humility to empathize with a fellow non-EA human being and recognize that the three features you've listed, are, in fact, everywhere. It is just that it takes much more than these three qualities to make a human.
I believe this recognition is essential for the future of the community and the psychological health of the citizens of this forum.
I want to talk to non-EA, not as to someone who doesn't share my values, but to someone who wasn't lucky enough to have a chance to make space for EA activities in their days, minds, and hearts but deep inside, we share the same honest EA idea "we aim to do good effectively, while also doing those other mysterious things that make us into a wholesome human being."
*This might be the very same way you feel. But I still thought it was important to share.
Great article.
I would suggest correcting the sentence "On the horizontal axis in each panel you see GDP per capita..." as what we see is a logarithm of the GDP per capita.
While this doesn't challenge any of the messages Max is making, one will get a very distorted perception if one assumes that all those metrics are linearly correlated with the GDPpc instead of the logarithm of the GDPpc. In practice, if we consider increasing GDPpc by 1000USD on the left side of the graphs it will shift the point by 30% of the whole range, while if we consider the reduction of the rightmost points by 1000 USD the shift will be ~1%.
This misreading happened to me on the first read and I have also heard other people saying that various metrics are correlated with the GDP, referring to this article.
Another even more nitpicky comment:
Although I understand that Max is simply reusing graphics from an external source, I should note that one might be deceived by the presentation in two other cases:
- in graphs 2 and 3 as the vertical axis is also logarithmic (implying power dependence on GDP).
- in the first and the last two graphs' Y-axis doesn't start from zero