was just part of PDKU and now back in Philly for my "normal job". I think this is important!
The problems in the bay (and maybe dc) and elsewhere seem very different. I think in the bay, there is enough critical mass that you could do e.g. sports league teams for the AI safety people. Exercise + social is good. Not sure if like constellation is already doing stuff like this, but maybe just pre pay for some different sports leagues and then distribute out flyers with easy signup to the various orgs and email some people. might punt a few k but would probably very little effort and decent upside. I played soccer with some AIS folks at a park and that was fun. I know I would always appreciate someone coming up to me and asking if I wanted to play on a sport team with like minded people for the next 10 weeks.
Outside the bay, idk man. If you can help start an IRL group for people looking to inform the public/gov or help organize existing EA/Rat group to do the same, that seems like a good way to channel the energy. But otherwise it's a bit crazymaking to even think about AIS at all because you will find so little support or understanding from everyone around you. The best thing you can probably do without going nuts is to set a reminder to call your senator and congress person once a month or two and otherwise delete twitter and don't think about it too much (unless you are working an actual AIS remote job, then idk).
In general I suppose I should have always felt the sadness associated with how bad the world is at triaging resources, that is sort of what EA is all about after all, but trying to help with AIS has made me feel this in such a visceral way that doesn't make me feel good compared to the animal welfare/poverty stuff. Maybe I had already more successfully compartmentalized those.
Agreed, I'd really want more like a RAND policy maker/rationalist to write out some theories of how people could do bio terrorism, and then grab the individual steps and ask viroligists. subject matter experts are usually quite myopic and can't see the bigger picture.
you might want to read these, the short is that it's not obvious that if we collapse the modern economy that we can actually get back to interstellar, because we will have way less non renewables (phosphorus, oil/coal) at our disposal next time (don't totally agree, but worth considering).
Point 1, strong agree esp w/ we don't know if per human (or human descendant) ev is positive or negative (relative to counterfactual, which could be nothing or aliens), wish this was more mainstream dogma here, not sure why longtermists think they can just not engage with this.
Point 3, again I'll re route to my response to point 2. Covid (the virus) wasn't even that bad in a sense and it was still catastrophic (for how much it affected society). Imagine something slightly worse than covid + a record heat wave that causes a massive refugee crisis or + a war. I'm not so confident this wouldn't massively collapse the global economy, and then route back to the fruit picking, we might only get 1-3 tries to go interstellar, so collapsing the global economy prob not == death but would == reduced chance of becoming grabby, which is ~= death from POV of total utilitarian.
We used to call this upper case (ea community/movement) vs lower case (the (meta) philosophy). I think meta normative framework/philsophy is the way I think about it; that is, you can apply EA to most normative frameworks, esp consequentalisty ones (though to many people in the community, the EA framework is actually just applied total utilitarianism). FWIW, and I say this as someone who has many issues with the EA community, the vast majority of EA criticisms from outside the community I see are just highly inaccurate and not worth engaging with from an intellectual POV (but maybe still worth engaging with for movement reputation).
Deceptive AIs will be able to hide unwanted behaviours from mechanistic interpretability tools (e.g. by encoding them redundantly across pathways, or shifting them into representations the tools do not capture) 8
I can trivially turn off or have fake thoughts running through the voice in my head. Subconscious brain activity seems harder but obviously you can manipulate that too by changing your surroundings and drugs and what not. I wouldn't be able to control those in a meaningful way nor do I think current AI's could (but wouldn't be shocked if they already could control the voice in their head if they have it). but I would guess future ai's will know how to control increasingly large parts of their brain activations.
At the margin, S-risk work in AI is more important than x-risk work³
From a utilitarian pov, It's not clear to me that the ev of the lightcone given we survive is positive (over nothing, or aliens, or life revolving on earth). From a humanist POV I'd rather focus on all of us surviving.
Theories of consciousness will lead to actionable understanding of AI consciousness²
Very bullish on there existing a mechanistic interpretation of consciousness (hard problem). I think it would follow that we would be able to understand if basically anything is conscious.
If animals continue to exist in a post-AGI world, animal suffering will not persist
I don't have a strong take on if agi or whoever is in control will be more moral than us but I'm guessing we will be a lot richer, and I think most likely whoever is in control won't want to torture anything (though they might not care much), and if we are alot richer and advanced I'd think this will spillover to better treatment of beings. I think chance of extreme digital suffering is much higher. The mostly like s-risk as I see it is of the hansonian mathusian version where you have expanders stuck in competition, but in this case idt there will be any or a morally relevant amount of animals
was just part of PDKU and now back in Philly for my "normal job". I think this is important!
The problems in the bay (and maybe dc) and elsewhere seem very different. I think in the bay, there is enough critical mass that you could do e.g. sports league teams for the AI safety people. Exercise + social is good. Not sure if like constellation is already doing stuff like this, but maybe just pre pay for some different sports leagues and then distribute out flyers with easy signup to the various orgs and email some people. might punt a few k but would probably very little effort and decent upside. I played soccer with some AIS folks at a park and that was fun. I know I would always appreciate someone coming up to me and asking if I wanted to play on a sport team with like minded people for the next 10 weeks.
Outside the bay, idk man. If you can help start an IRL group for people looking to inform the public/gov or help organize existing EA/Rat group to do the same, that seems like a good way to channel the energy. But otherwise it's a bit crazymaking to even think about AIS at all because you will find so little support or understanding from everyone around you. The best thing you can probably do without going nuts is to set a reminder to call your senator and congress person once a month or two and otherwise delete twitter and don't think about it too much (unless you are working an actual AIS remote job, then idk).
In general I suppose I should have always felt the sadness associated with how bad the world is at triaging resources, that is sort of what EA is all about after all, but trying to help with AIS has made me feel this in such a visceral way that doesn't make me feel good compared to the animal welfare/poverty stuff. Maybe I had already more successfully compartmentalized those.
https://forum.effectivealtruism.org/s/wmqLbtMMraAv5Gyqn
super dense and very well researched summary of some related things.
Agreed, I'd really want more like a RAND policy maker/rationalist to write out some theories of how people could do bio terrorism, and then grab the individual steps and ask viroligists. subject matter experts are usually quite myopic and can't see the bigger picture.
https://dianzhuo-wang.github.io/ fwiw this guy seems like he might have an interesting perspective.
Re point 2,
https://forum.effectivealtruism.org/posts/CxMusuX8E5hiTXEWX/fruit-picking-as-an-existential-risk
https://forum.effectivealtruism.org/posts/Nc9fCzjBKYDaDJGiX/what-is-the-likelihood-that-civilizational-collapse-would-1
you might want to read these, the short is that it's not obvious that if we collapse the modern economy that we can actually get back to interstellar, because we will have way less non renewables (phosphorus, oil/coal) at our disposal next time (don't totally agree, but worth considering).
Point 1, strong agree esp w/ we don't know if per human (or human descendant) ev is positive or negative (relative to counterfactual, which could be nothing or aliens), wish this was more mainstream dogma here, not sure why longtermists think they can just not engage with this.
Point 3, again I'll re route to my response to point 2. Covid (the virus) wasn't even that bad in a sense and it was still catastrophic (for how much it affected society). Imagine something slightly worse than covid + a record heat wave that causes a massive refugee crisis or + a war. I'm not so confident this wouldn't massively collapse the global economy, and then route back to the fruit picking, we might only get 1-3 tries to go interstellar, so collapsing the global economy prob not == death but would == reduced chance of becoming grabby, which is ~= death from POV of total utilitarian.
We used to call this upper case (ea community/movement) vs lower case (the (meta) philosophy). I think meta normative framework/philsophy is the way I think about it; that is, you can apply EA to most normative frameworks, esp consequentalisty ones (though to many people in the community, the EA framework is actually just applied total utilitarianism). FWIW, and I say this as someone who has many issues with the EA community, the vast majority of EA criticisms from outside the community I see are just highly inaccurate and not worth engaging with from an intellectual POV (but maybe still worth engaging with for movement reputation).
I can trivially turn off or have fake thoughts running through the voice in my head. Subconscious brain activity seems harder but obviously you can manipulate that too by changing your surroundings and drugs and what not. I wouldn't be able to control those in a meaningful way nor do I think current AI's could (but wouldn't be shocked if they already could control the voice in their head if they have it). but I would guess future ai's will know how to control increasingly large parts of their brain activations.
From a utilitarian pov, It's not clear to me that the ev of the lightcone given we survive is positive (over nothing, or aliens, or life revolving on earth). From a humanist POV I'd rather focus on all of us surviving.
Very bullish on there existing a mechanistic interpretation of consciousness (hard problem). I think it would follow that we would be able to understand if basically anything is conscious.
I don't feel confident at all, but the behavior of llms rn sure do remind me of at least elements of stress, discomfort, and happiness.
I don't have a strong take on if agi or whoever is in control will be more moral than us but I'm guessing we will be a lot richer, and I think most likely whoever is in control won't want to torture anything (though they might not care much), and if we are alot richer and advanced I'd think this will spillover to better treatment of beings. I think chance of extreme digital suffering is much higher. The mostly like s-risk as I see it is of the hansonian mathusian version where you have expanders stuck in competition, but in this case idt there will be any or a morally relevant amount of animals