Something roughly like this is also one of the main ways I bounce off. Especially with acausal stuff, I'm not sure I'm aware of any historical sign flips? Arguably the only people who drastically changed their actions because of acausal stuff are the Zizians.
(If you want to immediately skip to why I believe past attempts to incorporate LessWrong-style intuitions into standard decision theory are insufficient, go to footnote 16)
Ah yeah, maybe I should have made that clear, sorry. The original post that the list came from did, and I didn't think of that when I just copied the list over.
On your first sentence, not sure if you read footnote 3 - I believe I cover under the "value of the future" bullet point why I didn't include this. (combined with AIs probably being moral patients).
And your three bullet points, very briefly:
From a longtermist perspective, not important.
I consider permanent stagnation pretty unlikely for evolutionary reasons, eventually factions that want to grow will be selected for.
Yeah it's a factor, but AI just seems important enough that I highly doubt that it's a major one.
The original snippet was more about the acausal stuff, and wasn't Extrapolation, and the distinct argument I subsequently mentioned about simulators was Extrapolation.
There's no quick objection I can give to your response specifically (except to simplistically say "no, I don't buy it, we know more than that, we can have some reasonable guesses about simulators' intentions") - properly laying out my disagreements would take a little bit of time and effort, as is usually the case for deep worldview differences.
"But then we're back to the Extrapolation argument, which you claimed you weren't committed to."
No, I didn't claim that - I said the snippet you quoted wasn't the Extrapolation argument, and I stand by that. I'm definitely sympathetic to something like Extrapolation in general.
OK I read the LW comment you linked and skimmed the post, but I don't see how they show that we should expect lots of crucial considerations to come up specifically - they seem to argue more for "we're clueless about how much we should do ECL"? (but correct me if I'm wrong, I may have missed something). On your example, but why should I expect their attempts to do so to backfire if I don't already expect our own attempts to backfire? That seems like it just grounds out in the original debate. (btw, I also would like to get less confused about the similarity thing)
On the simulators, it just seems like its hard to think of possible simulator-motivations where us reaching good outcomes in the simulation would be bad for the base reality, and easy to think of ones where it would be neutral or good.
In general, I think the unawareness angle is genuinely interesting, I'm just not moved by it as much as you, probably for a few different reasons that would take some time to articulate.
I think it's not quite your "Extrapolation", because it's specifically about the acausal mechanism - by definition, the only (EDIT: direct) acausal effect possible is to make other agents take similar actions to us. (and then the simulation thing I kind of sweep under the rug because the footnote was quickly written, but the argument is somewhat similar (although very vague and I'd like something better): Whatever purpose our simulators have for simulating us, it's probably good for their reality too if we reach a good outcome in the simulation.)
As I said before, I think my biggest personal crux are the proposed downside risks (e.g. commitment races) of "promote wisdom, cooperation, altruism, etc" - I thought about them for a little bit and took some notes when i last read about it, but I remain skeptical that they can outweigh the intuitive upside. (I guess this is P3? But I'm not sure). Maybe I could write my thinking up at some point.
Something roughly like this is also one of the main ways I bounce off. Especially with acausal stuff, I'm not sure I'm aware of any historical sign flips? Arguably the only people who drastically changed their actions because of acausal stuff are the Zizians.
(If you want to immediately skip to why I believe past attempts to incorporate LessWrong-style intuitions into standard decision theory are insufficient, go to footnote 16)
Ah yeah, maybe I should have made that clear, sorry. The original post that the list came from did, and I didn't think of that when I just copied the list over.
On your first sentence, not sure if you read footnote 3 - I believe I cover under the "value of the future" bullet point why I didn't include this. (combined with AIs probably being moral patients).
And your three bullet points, very briefly:
Interesting stuff! I probably don't buy these as major worries but yeah, interesting.
The original snippet was more about the acausal stuff, and wasn't Extrapolation, and the distinct argument I subsequently mentioned about simulators was Extrapolation.
There's no quick objection I can give to your response specifically (except to simplistically say "no, I don't buy it, we know more than that, we can have some reasonable guesses about simulators' intentions") - properly laying out my disagreements would take a little bit of time and effort, as is usually the case for deep worldview differences.
"But then we're back to the Extrapolation argument, which you claimed you weren't committed to."
No, I didn't claim that - I said the snippet you quoted wasn't the Extrapolation argument, and I stand by that. I'm definitely sympathetic to something like Extrapolation in general.
OK I read the LW comment you linked and skimmed the post, but I don't see how they show that we should expect lots of crucial considerations to come up specifically - they seem to argue more for "we're clueless about how much we should do ECL"? (but correct me if I'm wrong, I may have missed something). On your example, but why should I expect their attempts to do so to backfire if I don't already expect our own attempts to backfire? That seems like it just grounds out in the original debate.
(btw, I also would like to get less confused about the similarity thing)
On the simulators, it just seems like its hard to think of possible simulator-motivations where us reaching good outcomes in the simulation would be bad for the base reality, and easy to think of ones where it would be neutral or good.
In general, I think the unawareness angle is genuinely interesting, I'm just not moved by it as much as you, probably for a few different reasons that would take some time to articulate.
I think it's not quite your "Extrapolation", because it's specifically about the acausal mechanism - by definition, the only (EDIT: direct) acausal effect possible is to make other agents take similar actions to us.
(and then the simulation thing I kind of sweep under the rug because the footnote was quickly written, but the argument is somewhat similar (although very vague and I'd like something better): Whatever purpose our simulators have for simulating us, it's probably good for their reality too if we reach a good outcome in the simulation.)
As I said before, I think my biggest personal crux are the proposed downside risks (e.g. commitment races) of "promote wisdom, cooperation, altruism, etc" - I thought about them for a little bit and took some notes when i last read about it, but I remain skeptical that they can outweigh the intuitive upside. (I guess this is P3? But I'm not sure). Maybe I could write my thinking up at some point.