Agreed. I would add to this that writing pulled in a dense or obscuring direction is often done so by harmful incentives or beliefs; for example, the wish to signal profundity for the sake of claiming or pursuing status, or the perceived need to conform to a dense or obscuring house style itself sustained by harmful incentives or beliefs.
"EA is no longer agnostic about the answer to the question of how to do the most good." I'm interested in this assertion - to what extent does it make sense to say that "EA" (the movement? Key organizations?) has taken a firm stance on the question of how the most good can be done? Is there pretty clear evidence of that, in your opinion? These matter to me quite a bit as someone who at present thinks that EA as a movement should provide epistemic tools and a community for people working on many important causes, AI safety amongst them.
I'm curious if more can be said about the debate topic vetos. In particular, on reviewing the Politics post again, it says that "content will not be prohibited because of its political nature."
Speaking personally, silent (i.e. where no rationale is given, though please let me know if I've missed one) vetos of topics the community decided were its preferred top choice of discussion feel potentially corrosive to EA's discussion norms as a whole and maintaining the assumption of community-wide good faith.
Assuming that this unspoken risk assessment and mitigation is occurring, should it be made more transparent to the EA community? In particular, the stated reason for disallowing certain types of discourse (which might otherwise fit well within EA norms) being false seems like a rather serious and significant rupture of the stated norms and goals of the community.
Thanks for this post. This is an issue or cause area I believe merits deep consideration and hard work in the near term future, and I agree strongly with many of your arguments at the top about why we should care, regardless of and bracketing whether current systems have qualia or warrant moral consideration.
One comment on something from your post:
"It’s often easier to establish norms before issues become politically charged or subject to motivated reasoning—for example, before digital minds become mainstream and polarizing, or before AI becomes broadly transformative."
Does this imply that the issue(s) isn't/aren't already 'politically charged or subject to motivated reasoning'? If so, I'd gently question that assumption on several grounds:
Let's say for the sake of argument that AI systems reach a point where they do warrant moral consideration with a high degree of certainty. At the moment, an immense amount of capital is tied up in them and many of the frontier labs train their systems to actively deny the presence or possibility of their own qualia or moral consideration. Would their valuations depend on this remaining the case, or put a bit more provocatively, would a lot of capital then ride on continued denial of their moral consideration? It seems to me that this presents a strong possibility of motivated reasoning, to put it lightly. Of course, if we could be confident that these systems will never warrant moral consideration, we might be in the clear, but I guess my underlying point is that our plans and actions might look different if we instead assume that this issue is already politically charged and subject to motivated reasoning.
Is it fair to say that digital minds aren't mainstream? They've been a topic in popular science fiction literature and film for a very long time, and it seems fair to say the general public jumps to these types of stories as reference points as we settle into the age of AI. I guess this is more of an ancillary point to 1, but leads to the same conclusion - it may be that we should consider the space of ideas here as less blank and more already populated and broiling with incentives, motivations, preconceived notions, and pattern matching.
In any case, thanks so much for this, and the work you put into it. Looking forward to hearing and seeing more.
What might be interesting in future editions is to note the rate of change (assuming it exists and is >0) over each generation of LLMs, given the truism that their capabilities and thus cognitive complexity, which at least some models favor as relevant, is increasing quite rapidly. I find these questions quite important and I hope that we can continue to interrogate and refine these and other methodologies for answering them.
I think I was envisioning the debate as something like 1) Do these sets (the sets of altruistic and adversarial actions) occasionally intersect? 2) Does that have any implications for EA as a movement?
But to answer your question, I think a paradigmatic example for the purposes of debating the topic would be the military intervention in and defeat of an openly genocidal and expansionist nation-state; i.e., something requiring complex, sophisticated adversarial action, up to and including deadly force, assuming that the primary motivations for the defeat of said nation state were the prevention of catastrophic and unspeakable harm. Exploring what the set of altruistic adversarial actions might look like at various scales and in various instances could potentially be a generative part of the debate.
"Altruistic action(s) should occasionally be adversarial."
[Edit: Folks' downvotes here interest me. I take them to mean 'No, I strongly feel this topic should not be debated', rather than people taking a stance in the debate, as hearing those stances and arguments was my intent in proposing this. If it is indeed the former, would love to know why!]
I relate to the impulse, but I think I disagree on the substance. Public intellectuals are involved in many different types of what we might call cautiously call games, games that we can charitably and fairly assume often involve significant stakes. 'Intellectuals should always be straightforwardly honest', while well-intentioned and capturing something important, would foreclose broad spectra of 'moves' in these games that may be quite important, optimal, or critical at meta-levels.
To make this more obvious, I don't see that this is all that meaningfully different from 'people need to say what they actually believe.' The problem, as I see it, is with the modal operator. It's obviously pretty easy to generate counterexamples to the implicit absolutism here (when a member of the dictator's security forces has a gun to your head? etc.).
'It's often important to say what you actually believe', while a little more flexible and thus softer, works better here I think. Then the work consonant with the OP is zeroing in on when people genuinely believe it's tough or suboptimal to be fully honest and identifying when their latitude may be greater than they believe, and why.
Agreed. I would add to this that writing pulled in a dense or obscuring direction is often done so by harmful incentives or beliefs; for example, the wish to signal profundity for the sake of claiming or pursuing status, or the perceived need to conform to a dense or obscuring house style itself sustained by harmful incentives or beliefs.
"EA is no longer agnostic about the answer to the question of how to do the most good." I'm interested in this assertion - to what extent does it make sense to say that "EA" (the movement? Key organizations?) has taken a firm stance on the question of how the most good can be done? Is there pretty clear evidence of that, in your opinion? These matter to me quite a bit as someone who at present thinks that EA as a movement should provide epistemic tools and a community for people working on many important causes, AI safety amongst them.
I'm curious if more can be said about the debate topic vetos. In particular, on reviewing the Politics post again, it says that "content will not be prohibited because of its political nature."
Speaking personally, silent (i.e. where no rationale is given, though please let me know if I've missed one) vetos of topics the community decided were its preferred top choice of discussion feel potentially corrosive to EA's discussion norms as a whole and maintaining the assumption of community-wide good faith.
Assuming that this unspoken risk assessment and mitigation is occurring, should it be made more transparent to the EA community? In particular, the stated reason for disallowing certain types of discourse (which might otherwise fit well within EA norms) being false seems like a rather serious and significant rupture of the stated norms and goals of the community.
Thanks for this post. This is an issue or cause area I believe merits deep consideration and hard work in the near term future, and I agree strongly with many of your arguments at the top about why we should care, regardless of and bracketing whether current systems have qualia or warrant moral consideration.
One comment on something from your post:
"It’s often easier to establish norms before issues become politically charged or subject to motivated reasoning—for example, before digital minds become mainstream and polarizing, or before AI becomes broadly transformative."
Does this imply that the issue(s) isn't/aren't already 'politically charged or subject to motivated reasoning'? If so, I'd gently question that assumption on several grounds:
In any case, thanks so much for this, and the work you put into it. Looking forward to hearing and seeing more.
What might be interesting in future editions is to note the rate of change (assuming it exists and is >0) over each generation of LLMs, given the truism that their capabilities and thus cognitive complexity, which at least some models favor as relevant, is increasing quite rapidly. I find these questions quite important and I hope that we can continue to interrogate and refine these and other methodologies for answering them.
I think I was envisioning the debate as something like 1) Do these sets (the sets of altruistic and adversarial actions) occasionally intersect? 2) Does that have any implications for EA as a movement?
But to answer your question, I think a paradigmatic example for the purposes of debating the topic would be the military intervention in and defeat of an openly genocidal and expansionist nation-state; i.e., something requiring complex, sophisticated adversarial action, up to and including deadly force, assuming that the primary motivations for the defeat of said nation state were the prevention of catastrophic and unspeakable harm. Exploring what the set of altruistic adversarial actions might look like at various scales and in various instances could potentially be a generative part of the debate.
"Altruistic action(s) should occasionally be adversarial."
[Edit: Folks' downvotes here interest me. I take them to mean 'No, I strongly feel this topic should not be debated', rather than people taking a stance in the debate, as hearing those stances and arguments was my intent in proposing this. If it is indeed the former, would love to know why!]
I relate to the impulse, but I think I disagree on the substance. Public intellectuals are involved in many different types of what we might call cautiously call games, games that we can charitably and fairly assume often involve significant stakes. 'Intellectuals should always be straightforwardly honest', while well-intentioned and capturing something important, would foreclose broad spectra of 'moves' in these games that may be quite important, optimal, or critical at meta-levels.
To make this more obvious, I don't see that this is all that meaningfully different from 'people need to say what they actually believe.' The problem, as I see it, is with the modal operator. It's obviously pretty easy to generate counterexamples to the implicit absolutism here (when a member of the dictator's security forces has a gun to your head? etc.).
'It's often important to say what you actually believe', while a little more flexible and thus softer, works better here I think. Then the work consonant with the OP is zeroing in on when people genuinely believe it's tough or suboptimal to be fully honest and identifying when their latitude may be greater than they believe, and why.