Thanks! This is super helpful. If the Opportunities Board has most of the part and full-time roles on 80k and PG's highlighted, then I struggle to imagine why I would recommend anything besides the Opportunity board, unless someone wanted to know about the opportunities that are less likely to be impactful by EA lights.
I suppose one reason I might recommend anything else, is that 80k's advisor bot integrates with the job board. Would be curious to hear from @Conor Barnes 🔶 and @Probably Good if I'm missing anything here.
Recommending lots of boards creates some friction in our 1-1s, but I may continue to recommend all 3 boards to most/all people if there are other reasons to do this.
I went to an ai ethics vs ai safety debate, my friend was speaking at yesterday in the Cambridge union
my summary of what happened is:
1. most people view negatively (1) Sam A starting openai to solve alignment faster than deepmind (2) Holden Karnofsky funding openai. some people, especially on ethics side, view negatively (3) Dustin Moskowitz investing in anthropic
2. the pattern of beliefs/behaviours that led to this could be characterised as (1) "thinking agi is inevitable", (2) "thinking we know best and can control things more than we can", (3) "believing AGI has a bunch of positives so it should be developed in some aligned sense" and (4) "attracting talented people who believe the above, maybe more than we do"
3. i think its reasonable to criticise any/all of those beliefs (Richard Ngo does so well in “retrospective on AI alignment”)
4. but the current situation makes it much harder to criticise the belief that AGI is inevitable/makes AGI not super useful as a term.
5. both sides agree on a bunch of governance interventions being on a great bet. accountability & transparency, auditing,
6. but people continue to disagree about how much we should support wholesale upheaval of capitalist structures that incentivise continued development, how possible pause is etc. Also the specific language people use is significantly different
1. 6 specifically is very hard to talk about without getting caught up in identities. but less of an identity war than 3.
also man the ethics vs safety debate was very unfun to watch. people far more focused on 3. and 6 than i’d like
I'd love to know how ProbablyGood fits into this! Would be helpful for my organising of EA Cambridge, as we currently refer people to job boards depending on their interests
As a mentee last year (and excited to be a mentor this year), I can't recommend OSP enough.
The post asks for concrete experiences, and I actually tried to quantify the value I was getting from my mentor about halfway through my first year. I ended up with this list of value-adds (the numbers were my personal, arbitrary "how much value did I get?" score):
Providing accountability - 15
Hero licensing/empowerment - 10
High-level strategy - 10
Low-level strategy - 10
Spot red flags/sign-off - 10
Being another EA (community/support) - 11
Problem-solving - 8
Talk through uncertainty - 9
...and a few others.
As a mentor I've found it really fulfilling to help others receive mentorship! Looking forward to mentoring you if you apply!
On CGP Grey, he has 6.8M YouTube subs and seems to get the alignment concerns. He recently conveyed the alignment risk in this episode of his tech podcast (Revisiting Humans Need not Apply):
"AI is more like biological weapons because they can act autonomously and evolve beyond what you built. Nuclear bombs don't walk out of factories on their own, pathogens do."
Might be worth someone reaching out about e.g. sponsorship.
quick ideas (some that i didn't actually implement v well but are apparently good for lots of ppl): - Time blocking parts of the day or week - Being really serious about not prioritising exciting projects if you feel rly stretched, asking yourself questions like "would it be less impactful if i handed this to next year's committee" or "if i work on this, what am i not able to do" - Keeping in mind this advice about doing 1 or 2 projects at time. I think this applies to e.g. if you have 2 clubs to run. But, i think this applies less if 1 project is sport, and the other is EA - doing events I personally looked forward to. I spent ~20% of my EA time 1-1ing with people cause i really enjoyed it. also running events that you personally enjoy is a good(?) heuristic for others enjoying it
Thanks for this! I think this kind of post is hugely valuable! If you (yes you the reader!) know someone who has skills/interest in this area, consider them sending this post! I've done the same for a friend who I'd previously assumed was "lost" to the corporate ladder. They've seen this post and they're now excited to chat to you Patrick :).
If you need extra convincing that reaching out to your sales friends with this post is a good idea, (or maybe you should write your own post along these lines):
- As a movement we're currently bottle necked by experienced professionals and this kind of post seems hugely valuable to encourage sales people to use their skills for good.
- I also think most cause areas would benefit from a more diverse set of backgrounds, skills and mindsets, and as the top cause areas mature this can happen!
This comment prompted a lot of reflection, so thank you!
I don't think the blog post claims that LLMs are a long way from human-like intelligence. For what it's worth, I agree with your reasoning against that line of argument.
My main takeaway from this post's core about being mindful about the level of abstraction in language. Obviously for technical AI safety, the low-level, mechanistic view seems important. But it also seems like rhetorical sleight of hand to go with high-level anthropomorphic language to motivate people/make explanations easier. In good written up resources they lead with a fundamental understanding of e.g. how neural nets work (bluedot, global challenges project) but I personally think the movement could still bear this in mind more when introducing AI safety to newcomers. Needless to say hype language is also a problem in mainstream capabilities discussion.
Side note: On the analogy to physics itself, I'm not an expert, but I've also been told that the premise of the universe or brain being describable by purely linear maps is contested. Regardless of that, I'm not sure how pragmatically important the analogy is compared to the immediate choice of which abstraction to use for AI safety work.
Thanks! This is super helpful. If the Opportunities Board has most of the part and full-time roles on 80k and PG's highlighted, then I struggle to imagine why I would recommend anything besides the Opportunity board, unless someone wanted to know about the opportunities that are less likely to be impactful by EA lights.
I suppose one reason I might recommend anything else, is that 80k's advisor bot integrates with the job board. Would be curious to hear from @Conor Barnes 🔶 and @Probably Good if I'm missing anything here.
Recommending lots of boards creates some friction in our 1-1s, but I may continue to recommend all 3 boards to most/all people if there are other reasons to do this.
I went to an ai ethics vs ai safety debate, my friend was speaking at yesterday in the Cambridge union
my summary of what happened is:
1. most people view negatively (1) Sam A starting openai to solve alignment faster than deepmind (2) Holden Karnofsky funding openai. some people, especially on ethics side, view negatively (3) Dustin Moskowitz investing in anthropic
2. the pattern of beliefs/behaviours that led to this could be characterised as (1) "thinking agi is inevitable", (2) "thinking we know best and can control things more than we can", (3) "believing AGI has a bunch of positives so it should be developed in some aligned sense" and (4) "attracting talented people who believe the above, maybe more than we do"
3. i think its reasonable to criticise any/all of those beliefs (Richard Ngo does so well in “retrospective on AI alignment”)
4. but the current situation makes it much harder to criticise the belief that AGI is inevitable/makes AGI not super useful as a term.
5. both sides agree on a bunch of governance interventions being on a great bet. accountability & transparency, auditing,
6. but people continue to disagree about how much we should support wholesale upheaval of capitalist structures that incentivise continued development, how possible pause is etc. Also the specific language people use is significantly different
1. 6 specifically is very hard to talk about without getting caught up in identities. but less of an identity war than 3.
also man the ethics vs safety debate was very unfun to watch. people far more focused on 3. and 6 than i’d like
I'd love to know how ProbablyGood fits into this! Would be helpful for my organising of EA Cambridge, as we currently refer people to job boards depending on their interests
Wow you can even make polls in a comment!
As a mentee last year (and excited to be a mentor this year), I can't recommend OSP enough.
The post asks for concrete experiences, and I actually tried to quantify the value I was getting from my mentor about halfway through my first year. I ended up with this list of value-adds (the numbers were my personal, arbitrary "how much value did I get?" score):
Providing accountability - 15
Hero licensing/empowerment - 10
High-level strategy - 10
Low-level strategy - 10
Spot red flags/sign-off - 10
Being another EA (community/support) - 11
Problem-solving - 8
Talk through uncertainty - 9
...and a few others.
As a mentor I've found it really fulfilling to help others receive mentorship! Looking forward to mentoring you if you apply!
On CGP Grey, he has 6.8M YouTube subs and seems to get the alignment concerns. He recently conveyed the alignment risk in this episode of his tech podcast (Revisiting Humans Need not Apply):
"AI is more like biological weapons because they can act autonomously and evolve beyond what you built. Nuclear bombs don't walk out of factories on their own, pathogens do."
Might be worth someone reaching out about e.g. sponsorship.
Super appreciate this kind of narrative retrospective :)
Excited to come next year!
quick ideas (some that i didn't actually implement v well but are apparently good for lots of ppl):
- Time blocking parts of the day or week
- Being really serious about not prioritising exciting projects if you feel rly stretched, asking yourself questions like "would it be less impactful if i handed this to next year's committee" or "if i work on this, what am i not able to do"
- Keeping in mind this advice about doing 1 or 2 projects at time. I think this applies to e.g. if you have 2 clubs to run. But, i think this applies less if 1 project is sport, and the other is EA
- doing events I personally looked forward to. I spent ~20% of my EA time 1-1ing with people cause i really enjoyed it. also running events that you personally enjoy is a good(?) heuristic for others enjoying it
Thanks for this! I think this kind of post is hugely valuable! If you (yes you the reader!) know someone who has skills/interest in this area, consider them sending this post! I've done the same for a friend who I'd previously assumed was "lost" to the corporate ladder. They've seen this post and they're now excited to chat to you Patrick :).
If you need extra convincing that reaching out to your sales friends with this post is a good idea, (or maybe you should write your own post along these lines):
- As a movement we're currently bottle necked by experienced professionals and this kind of post seems hugely valuable to encourage sales people to use their skills for good.
- I also think most cause areas would benefit from a more diverse set of backgrounds, skills and mindsets, and as the top cause areas mature this can happen!
This comment prompted a lot of reflection, so thank you!
I don't think the blog post claims that LLMs are a long way from human-like intelligence. For what it's worth, I agree with your reasoning against that line of argument.
My main takeaway from this post's core about being mindful about the level of abstraction in language. Obviously for technical AI safety, the low-level, mechanistic view seems important. But it also seems like rhetorical sleight of hand to go with high-level anthropomorphic language to motivate people/make explanations easier. In good written up resources they lead with a fundamental understanding of e.g. how neural nets work (bluedot, global challenges project) but I personally think the movement could still bear this in mind more when introducing AI safety to newcomers. Needless to say hype language is also a problem in mainstream capabilities discussion.
Side note: On the analogy to physics itself, I'm not an expert, but I've also been told that the premise of the universe or brain being describable by purely linear maps is contested. Regardless of that, I'm not sure how pragmatically important the analogy is compared to the immediate choice of which abstraction to use for AI safety work.