I suspect that the way most people (not just people with high leverage) interact with LLMs today is more relevant to catastrophic risk than it first appears to be.

• I worry that the way most people interact with LLMs today is writing the script for how we will interact with increasingly intelligent AIs going forward.
• If ~ASI is close, we should be identifying a rough optimal idea of what a human’s relationship with an LLM should look like (eg preserving critical thinking, thinking things through oneself, aiming to understand something rather than have the LLM do the thinking/understanding for you) in order to avoid ‘locking in’ dynamics that could enable gradual disempowerment.
• Although how people with the most leverage are interacting with current AI systems is probably the most important factor here (eg people with decisionmaking power in society), I think that it's important we foster a culture more broadly which values human autonomy, especially since each insitution in society likely faces unique risks around how it interacts with AI.
• To be clear, if alignment and superhuman intelligence becomes proven over time then I do think some degree of gradual handover of agency on important issues is likely overall value-positive. But I don't think we're at that stage.