I agree and made a similar claim previously. While I believe that many currently effective interventions are neglected, I worry that there are many potential interventions that could be highly effective but are overlooked because they are in cause areas not seen as neglected.
I have no views on whether NAO should be higher ranked, just noting that some of this is currently being done by governments, e.g. the UK Government has recently unveiled plans for an early warning system for pandemics: https://www.gov.uk/government/news/uk-to-create-world-first-early-warning-system-for-pandemics
Thank you for writing this well-argued post - I think its important to keep discussing exactly how big P(doom) is. However, and I say this as someone who believes that P(doom) is on the lower end, it would also be good to be clear about what the implications would be for EAs if P(doom) was low. It seems likely that many of the same recommendations - reduce spending on risky AI technologies and increase spending on AI safety - would still hold, at least until we get a clearer idea of the exact nature of AI risks.
How dependent is current AGI safety work on deep RL? Recently there has been a lot of emphasis on advances in deep RL (and ML more generally), so it would be interesting to know what the implications would be if it turns out that this particular paradigm cannot lead to AGI.
In general, I don't spend much time worrying about "neglectedness" in the ITN framework. I think that is because while many important problems are not neglected in an absolute sense (e.g. climate change), it is still the case that some solutions will be much more effective than others within each problem area. Therefore, one may have a large impact on less neglected problems by simply focusing on more effective solutions.
Don't try to wake up and save the world. Don't be bycatch. Take 15 years and become a domain expert. Take a career and become a macrostrategy expert. Mentor. Run small and non-EA projects. Circle back to EA periodically with your newfound skills and see what a difference you can make then. There is absolutely no way we can have a longtermist movement if we can't be longtermist about our own lives and careers. But if we can, then we can.
Many social movements I have been a part of (political, sports, religious etc.) have a sort of "more is more" aspect to them that I see a lot in EA. The basic idea is this: it is fine if you just want try things out, but the more involved you are in the organization, the more events you go to, the more you align your life around the mission of the organization, the better. To a large part, this is why I have often experienced "organizational burnout": when due to changing circumstances I cannot or do not want to be as involved anymore, it is often easier to quit altogether rather than to scale things back. With most of my commitments these days I try to follow a "less is more" approach, where I try to capture about ~10% of the scope of an organization or movement. The 10% scope of EA seems to be the thinking that we should consider how important a problem is when choosing a career or where to donate our funds. If we get more involved than this, there is a danger of not being able to sustain it over a long time. The other advantage of being minimally involved is that we can join a larger number of organizations and get a diverse and rich set of benefits that exceed what any one organization can provide.
PhDs get a lot of negative hype these days, so much that I wonder if it is potentially underrated it as a viable career step. I am just coming out of a PhD programme in the UK, and while I didn't always enjoy my topic nor want to continue research in my field, I still think it is overall a positive experience. It is important to realize that most people starting PhDs are well aware of the low chances of becoming a professor, but luckily there are still many good career options outside academia (however I do think all the negative hype around academic careers probably means that it is also underrated relative to its true value). Some of the good aspects of (science) PhDs are:
- Lots of flexibility. You are basically guaranteed an income for ~4-5 years that you cannot lose no matter how poorly you do. While your supervisor certainly has some influence over what you do during this time, you are surprisingly free to work on what you want. - High potential for growth. You get to experience just how difficult it is to become a world expert in something, albeit very niche, because most often you are the first to attempt your line of research. Learning about what has been done before in a field, how to find a viable approach, and how to overcome unexpected setbacks are very transferable skills. - This is a bit more speculative, but I'd reckon that you have higher chances of landing an intellectually stimulating career with a PhD compared to a Master's degree, at least in fields where credentials matter. I'd also reckon that as a PhD plus a few years experience you are likely to get promoted to leadership positions faster. - Many fields EAs care about (nuclear security, biosafety, AI etc.) are very science-heavy fields where having a PhD is useful. For example, with a Physics PhD it would be relatively easy to get a job in either of these, since your skillset is sufficiently adaptable to most analytic fields.
Thanks for sharing your perspective! It seems like having a family in major metropolitan areas are especially challenging due to the much higher housing cost. I am wondering if you have any examples of the types of jobs you think would be difficult to afford raising a family in London (alternatively, what salary)? For example, it seems that a civil servant could earn £40,000 per year after a few years of experience, and I suspect other sectors where EAs would want to work might pay a similar amount (academia, NGOs etc).
Regarding having lots of time, it is true that being a stay at home parent leads to substantial loss of income. What I was wondering was more along the lines of: is it worth trying to earn say £80,000+ per year working in finance just to be able to afford a larger house, but working 80+hours/week, when say a civil servant would have fixed 40 working hours per week, free weekends, but earning half as much. In terms of income vs time, my intuition is that time is more valuable than income when having children, even if it means saving on housing costs.
A big part of many peoples motivation for earning a high income seems to be the perception that it is a necessity in order to raise a family. Many EA-aligned jobs are in the public or NGO sector and are less paid than what people could earn in the private sector, and since close to 80% people have children, this could be big factor for people to give up on an EA-aligned career.
I am wondering whether this reasoning is valid, and where the extra cost for children comes from. In most western countries, there is high quality education available freely for everyone, healthcare is either free or subsidized, and children generally receive lots of support from the government (free dentist appointments, free/subsidized school meals, student loans etc). Food is a small part of the monthly budget and shouldn't be a big factor, clothes could be obtained from low-cost outlets or second-hand. That leaves more expensive budget items such as having a big house, a large car, vacation money etc. However, these might just be luxuries that make a relatively small difference in how "successful" one is in raising a family compared to other things, such as having lots of time.
It would be really interesting to see some analysis on how the decision about whether to have children or not should impact one's career planning, especially for those considering EA-aligned career options.
I agree and made a similar claim previously. While I believe that many currently effective interventions are neglected, I worry that there are many potential interventions that could be highly effective but are overlooked because they are in cause areas not seen as neglected.
Another potential source of funds is the UK Government's estimated ~4 billion GBP worth of bitcoin holdings
I have no views on whether NAO should be higher ranked, just noting that some of this is currently being done by governments, e.g. the UK Government has recently unveiled plans for an early warning system for pandemics: https://www.gov.uk/government/news/uk-to-create-world-first-early-warning-system-for-pandemics
Thank you for writing this well-argued post - I think its important to keep discussing exactly how big P(doom) is. However, and I say this as someone who believes that P(doom) is on the lower end, it would also be good to be clear about what the implications would be for EAs if P(doom) was low. It seems likely that many of the same recommendations - reduce spending on risky AI technologies and increase spending on AI safety - would still hold, at least until we get a clearer idea of the exact nature of AI risks.
How dependent is current AGI safety work on deep RL? Recently there has been a lot of emphasis on advances in deep RL (and ML more generally), so it would be interesting to know what the implications would be if it turns out that this particular paradigm cannot lead to AGI.
In general, I don't spend much time worrying about "neglectedness" in the ITN framework. I think that is because while many important problems are not neglected in an absolute sense (e.g. climate change), it is still the case that some solutions will be much more effective than others within each problem area. Therefore, one may have a large impact on less neglected problems by simply focusing on more effective solutions.
Many social movements I have been a part of (political, sports, religious etc.) have a sort of "more is more" aspect to them that I see a lot in EA. The basic idea is this: it is fine if you just want try things out, but the more involved you are in the organization, the more events you go to, the more you align your life around the mission of the organization, the better. To a large part, this is why I have often experienced "organizational burnout": when due to changing circumstances I cannot or do not want to be as involved anymore, it is often easier to quit altogether rather than to scale things back. With most of my commitments these days I try to follow a "less is more" approach, where I try to capture about ~10% of the scope of an organization or movement. The 10% scope of EA seems to be the thinking that we should consider how important a problem is when choosing a career or where to donate our funds. If we get more involved than this, there is a danger of not being able to sustain it over a long time. The other advantage of being minimally involved is that we can join a larger number of organizations and get a diverse and rich set of benefits that exceed what any one organization can provide.
PhDs get a lot of negative hype these days, so much that I wonder if it is potentially underrated it as a viable career step. I am just coming out of a PhD programme in the UK, and while I didn't always enjoy my topic nor want to continue research in my field, I still think it is overall a positive experience. It is important to realize that most people starting PhDs are well aware of the low chances of becoming a professor, but luckily there are still many good career options outside academia (however I do think all the negative hype around academic careers probably means that it is also underrated relative to its true value). Some of the good aspects of (science) PhDs are:
- Lots of flexibility. You are basically guaranteed an income for ~4-5 years that you cannot lose no matter how poorly you do. While your supervisor certainly has some influence over what you do during this time, you are surprisingly free to work on what you want.
- High potential for growth. You get to experience just how difficult it is to become a world expert in something, albeit very niche, because most often you are the first to attempt your line of research. Learning about what has been done before in a field, how to find a viable approach, and how to overcome unexpected setbacks are very transferable skills.
- This is a bit more speculative, but I'd reckon that you have higher chances of landing an intellectually stimulating career with a PhD compared to a Master's degree, at least in fields where credentials matter. I'd also reckon that as a PhD plus a few years experience you are likely to get promoted to leadership positions faster.
- Many fields EAs care about (nuclear security, biosafety, AI etc.) are very science-heavy fields where having a PhD is useful. For example, with a Physics PhD it would be relatively easy to get a job in either of these, since your skillset is sufficiently adaptable to most analytic fields.
Thanks for sharing your perspective! It seems like having a family in major metropolitan areas are especially challenging due to the much higher housing cost. I am wondering if you have any examples of the types of jobs you think would be difficult to afford raising a family in London (alternatively, what salary)? For example, it seems that a civil servant could earn £40,000 per year after a few years of experience, and I suspect other sectors where EAs would want to work might pay a similar amount (academia, NGOs etc).
Regarding having lots of time, it is true that being a stay at home parent leads to substantial loss of income. What I was wondering was more along the lines of: is it worth trying to earn say £80,000+ per year working in finance just to be able to afford a larger house, but working 80+hours/week, when say a civil servant would have fixed 40 working hours per week, free weekends, but earning half as much. In terms of income vs time, my intuition is that time is more valuable than income when having children, even if it means saving on housing costs.
How much money is required to raise a family?
A big part of many peoples motivation for earning a high income seems to be the perception that it is a necessity in order to raise a family. Many EA-aligned jobs are in the public or NGO sector and are less paid than what people could earn in the private sector, and since close to 80% people have children, this could be big factor for people to give up on an EA-aligned career.
I am wondering whether this reasoning is valid, and where the extra cost for children comes from. In most western countries, there is high quality education available freely for everyone, healthcare is either free or subsidized, and children generally receive lots of support from the government (free dentist appointments, free/subsidized school meals, student loans etc). Food is a small part of the monthly budget and shouldn't be a big factor, clothes could be obtained from low-cost outlets or second-hand. That leaves more expensive budget items such as having a big house, a large car, vacation money etc. However, these might just be luxuries that make a relatively small difference in how "successful" one is in raising a family compared to other things, such as having lots of time.
It would be really interesting to see some analysis on how the decision about whether to have children or not should impact one's career planning, especially for those considering EA-aligned career options.