zchuang's Quick takes

I think an inviting form of decoupling norms where it's fractured in chains. I don't think decoupling norms work when both parties don't opt-in and so people should switch to the dominant norm of the sphere. An illustrative example is as follows:

1. EAs try to avoid being persuasive and be explanatory and thus do not engage within the frame of opponents for fear of low fidelity. E.g. Will Macaskill sweatshop discourse about impermissibility and instead contest the claim on the metaethical level.
2.  EAs then respond that utilitarianism is true and people aren't engaging or justifying on the comparative (which is a category mistake about morality for a deontologist) and act as if the critic was non-responsive (the Chappell thread comes to mind).
   1. I will note this is a special case in which many EA critics don't want to re-litigate the utilitarianism vs. deontology discourse but want to articulate the side constraint violation and inform others of that thinking in EA.
3. The critic feels misunderstood because the EAs are very nice and say they're really happy the critic is here but the critic doesn't feel actually heard because the criticism wasn't really responded to but EAs say nice words of praise.
4. In turn the critic continues on the same chain of logic that EAs have not sharpened and pushed towards the true crux.

Some EAs would see this as being a motte-and-bailey instead of getting to the crux but cruxes can be asymmetric in that different critics combine claims together (e.g. the "woke" combining with more centrist sensibilities deontologists). But I think explanations which are done well are persuasive because they reframe truth-seeking ideas within accessible language that dissolve cruxes to seek agreement and cooperation. 

Another illustration on the macro-level of the comparative:

1. Treat critics well on the resource and out of argumentative level such that the asymmetric resources of EA don't come to bear.
2. Make sure you are responsive to their concerns and use reasoning transparency. Instead of saying, "thanks for responding and being a critic" and leaving it there actually engage forcefully with the ideas and then thank them for their time. 
3. The core problem right now is that EAs lead with being open to change their mind which sets up discourse for failure because when EAs don't change their minds the critics feel misled. 

To be clear, there are harms with trying to be persuasive (e.g. sophistry, lying, motivated reasoning etc.). But sometimes being persuasive is about speaking the argumentative language of another side.

But that's conditional on people not interpreting the hugboxing as a tactic/weird norm. E.g. mormon missionaries being nice to people doesn't elicit the same response as a person off the street because they adjust their set point.

Great question that prompted a lot of thinking. I think my internal model looks like this:

1. On the meta level it feels as if EAs have a systemic error in their model that underestimates public distrust of EA actions which constrains the action space and our collective sense-making of the world. 
2. I think legacy media organisations buy into the framing solidly. Especially, organisations that operate on policing others such as the CJR (Columbia Journalism Review). 
3. Just in my own life I've noticed a lot of the "elite" sphere friends I have at ivies and competitive debating etc. are much more apprehensive towards EA and AI Safety types of discourse in general and attribute it to this frame. Specifically, I think the idea from policy debating of inherency -- that people look towards frames of explaining the underlying barrier and motivation to change.
   1. I think directly this is bad for cooperation on the governance side (e.g. a lot of the good research on timelines and regulation are currently being done by some people with AI Ethics sympathies).
   2. I think EAs underestimate how many technically gifted people who could be doing technical research are put off by EAs who throw around philosophy ideas that are ungrounded in technical acumen. This frame neatly compounds this aversion.

I always thought the average model for don't let AI Safety enter the mainstream was something like (1) you'll lose credibility and be called a loon and (2) it'll drive race dynamics and salience. Instead, I think the argument that AI Ethics makes is "these people aren't so much loons as they are just doing hype marketing for AI products in the status quo and draining counterfactual political capital from real near term harms".

Here I'll illustrate the problem with your viewpoint on the ambiguity problem I have. Just going to spitball a bunch of problems I end up asking myself.

1. If I'm steelmanning the viewpoint:
   1. I'd be defending how this bundle of reasons leads to the conclusion. Conceptually easy and normal steelmanning.
   2. I'd provide strong evidence for each claim and my burden would be showing how load bearing of a claim and how it's mistaken or how it could reasonably come to be.
2. If I'm steelmanning a world view I'd be defending something like this follows:
   1. Am I defending the modal reasoning of the viewpoint holder across a population or the top reason holder (e.g. the average deontologist or Korsgaard?) 
   2. Am I defending the top world view holder in the eyes of the steelman opponent or in some objective modal observer (e.g. if I'm saying p(doom) = 90% is wrong do you want me to Paul Christiano's viewpoint or Yann LeCunn's)
      1. Subquestion here is do you want some expected value calculus of: change from base opinion * likelihood to change opinion.
3. If I'm steelmanning assumptions:
   1. Do you want the assumptions to line up with the opposing viewpoint such that the criticism of assumptions dissolves?
   2. Do you want me to make explicit the assumptions and provide broad reasons for them?
4. Are you steelmanning a specific reason:
   1. E.g. for steelmanning bioanchors we could decompose it down to inputs, probability distribution problems, forecasting certainty. Each of these argues from an external or internal frame of the bioanchors report and shows differing levels of understanding -- especially because they're interlinked. For instance, if you wanted a steelman of long-timelines|fast take off you end forcing a hardware overhang argument.
5. Are you steelmanning a reconstruction:
   1. Arguments often exist in chains of argumentation so steelmans that isolate and so a reconstructive defence of an argument that asks for a steelman is often decontextualised and confused.

Overall, I think a large part of the problem is the phrase "what is your steelman" being goodharted in the same way "I notice I'm confused" is where the original meaning is loss in a miasma of eternal September.