There are many other human practices that bring about a regression to the mean in writing. In fact, most all education steers toward this, rather than away. Of course, as writing is for communication, a high degree of regression is inherently necessary for function. If language was random, rather than common, it would be non-functional.

Now, we still do value creativity between these norms, but is this argument that AI editing regresses to a mean more a reflection of a regression toward a shared communication mechanism or a regression of creative elements, like ideas or high level concepts?

I would find it very interesting if someone had asked that question before this phrase of "regression to the mean" became globally accepted as a description of a result of AI writing. The phrase may be mathematically correct, but the connotation attached to it is the second, while logically the first is probably predominant. The first is simpler and more basic. I'd expect the simpler and more basic factor to be more rapidly regressed. The second is more complex. To regress the second, the training process would first have to form representations of the concepts and ideas to regress toward. It should come later.

Also, keep in mind that "regression to the mean" in mathematics accepts a connotational conflict with "regress" more generally in English. In general use, there is progress and regress, and regress carries the negative connotation. "Regression to the mean" swallows both the progress and regress as one movement. You still might dislike a regression to the mean if you value the loss of the peaks more than the filling of the valleys, but you might wonder, if the peaks are simply held out, and not shaved off, is there loss at all?

I don't want to fully dismiss such concerns, I consider them unresolved. But, I do want to disrupt the consensus that has formed by a lazy acceptance of false premises.

Of course the best mechanism would be a test, but as far as I know, no one using that phrase has even considered such a test. So I imagine it will fall upon those who do not support it's use to do that work.

Examples of human practices that regress writing style, just to hit the tip of the iceberg:

• Active voice
• Confident writing
• Study writers you like

Honestly, in many domains, we should just leave it up to the author to decide and not care. If they think the output is better than the input, and we can identify an idea or concept that's valuable inside, we shouldn't care what any tool says.

This isn't to excuse many ways of dishonestly seeking status or wasting people's time with meaningless drivel (two practices that commonly coincide). And yes, the easiest of all possible ways to do both of those is "Write a 2,000 word post arguing for my opinion being right / my product being the best", but it's merely at the top of a long list.

I think this is a reasonable response, but Cowen did anticipate the "slavery is immoral" response, and is right that this wouldn't be a utilitarian response.  You can fix that since there is an easily drawn line from utilitarianism to this response, but I think Cowen would respond that in this scenario we both wouldn't and shouldn't bother to do such fine reasoning and just accept our partialities.  He does make a similar statement during the Q&A.

I'd contend that this an example of mixing practical considerations with philosophical considerations. Of course we wouldn't stop during an invasion of little green men who are killing and enslaving humans and wonder.. "would it be better for them to win?"  If you did stop to wonder, there might be many good reasons to say no, but if you're asking a question of whether you'd stop and ask a question, it's not a philosophical question anymore, or at least not a thought experiment.  Timing is practical not theoretical. 

If it was really all about partialities, and not practical, it wouldn't matter what side we were on. If we showed up on another planet, and could enslave/exterminate a bunch of little green men, should we stop to think about it before we did?  Of course we should.  And while maybe you can concoct a scenario in which it's kill or be killed, there would be little question about the necessity to be certain that it wasn't an option to simply turn around and go the other way. 

Cowen would benefit by understanding that EA has to have a duality, encompassing both philosophical aspects and practical aspects.  The goal of EA is to have a practical effect on the world, but it's effect will be limited if it's not persuasive.  The persuasive aspect requires it to describe it's philosophical underpinnings.  It cannot rely solely upon a pre-existing partial point of view, because it's a departure from pre-existing partial points of view.  You might call it a refinement of reciprocal altruism which would be a widely shared pre-existing partial point of view that is not individualistic, that escapes some of our other biases that limit the scope to which that point of view is applied.

That said, since EA does not try to force or coerce, it is left with persuasiveness. Persuasiveness must be rooted in something, and philosophy is where all such roots begin for individuals  if they are not natural or imposed by force.  To disambiguate the preceding, it's not formal philosophy I refer to, but the broader scope philosophy which every individual engages with and is what formal philosophy attempts to describe and discuss.

The problem with the way Cowen engages with trolley problems and repugnant conclusions is that these are formal questions, out of formal philosophy, which are then forced into conflict with the practical world.  The difference between someone who flips the trolley lever or accepts infinite double or nothing 51% bets, and someone who doesn't, is whether they answer this as a formal question, or do not.

As a formal question, everything practical is removed.  That is not possible in a practical environment. Hubris, is necessary to accepting 51% double of nothing bets.  Hubris, is what would have prevented FTX's path, not a different answer to a formal philosophical question.

You can actually feel Cowen is drawn toward this answer, but continue to reject it.  He doesn't want EA to change, because he recognizes the value it provides.  At the same time, he's saying it's wrong and should be more socially conservative.

I'd disagree with the specifically socially conservative aspect, but agree with the conservative aspect. In this case I mean conservative in it's most original form.. cautious about change.  That said, I'm not convinced that the EA movement overall is not conservative, and I would not agree that "social conservatives" are all that conservative.  The typical "social conservative" is willing to make grand unsubstantiated statements, advocate for some truly repressive actions to enforce maintaining a prior social order (or reverting to one long since abandoned).

Being socially conservative does not make you conservative.  This narrow form of conservatism, can put you in conflict with other aspects of conservatism, and I'd argue it has put today's social conservatives at great odds with it.  In addition, there are large groups of people we allow to use the social conservative label who are regressive.  Regressivity is not conservative, as it's no longer attempting to maintain the status quo.  An attempt to regress will without a doubt have unintended consequences.  It's the existence of unintended consequences and the hubris to accept that you cannot see all of them that is the only value proposition to conservatism, so once this is abandoned there's no value left and we really shouldn't accept the use of conservative to describe such groups.

So, in short, EA should continue to engage on it's formal side, but should also continue to embrace practical principles during the translation of formal ideas to practical reality.  If an alien does come to us and offer endless 51% double or nothing bets, we should be skeptical of the honesty, ability, and all the margins that go with that around accepting that bet.  In like ways, when operating a financial company, someone should accept the possibility of downside, the inability to forecast the future, the reality of margin calls, the sometimes inscrutable wisdom of regulation, and the downsides of disregarding rules, even when it seems (to us) like we know better.

Forcing EA to divorce it's practical and philosophical sides and then attacking the philosophical side using practical arguments is either dishonest, or a failure to understand EA.  Accepting "partial point of view" as an absolute, provides no room for EA to argue for any change.  Conservatism may have a place, but it would be internally inconsistent if it ever was all encompassing, because the one thing that has always been constant, is change.  Conservatism that rejects change, rather than simply applying some caution to it, posits an impossibility, change cannot be halted, only managed.

Interesting piece. One challenge in extending it to decision making is "resources". It's not clear if you mean financial instruments or some kind of stockpiling. There appears to be some not fully considered vacillation on that topic.

Financial instruments are probably the default, but as we move into more and more long term views, the meaning of these becomes more vague. Does it really pass "resources" to a future generation by having stockpiled financial instruments? While in a micro-economic level these are very translatable to actual resources, at the macro-level, the clarity of that relationship breaks down. Printing a trillion extra dollars doesn't increase available resources, it merely shifts the locus of acknowledged control over the existing resource pool.

Likewise, if a long term philanthropist stockpiled financial instruments to be released in 500 years, keeping them fully dormant, they wouldn't transport physical resources 500 years into the future. What it would do is to create a shock to the system in 500 years of a new source of control. During the 500 years where these instruments were dormant, the rest of society would likely behave as if they did not exist, using all available physical resources during the dormant period without any stockpiling.

In addition, many financial instruments aren't dormant by their very nature, but directive. Investment in a stock directs resources and is an active influence. If this follows societal norms it would have little impact, but also not shift resources into the future.

What it can do is shift influence within the future. If that's a valuable enough goal, you still have to consider hazards. If the future isn't willing to accept this influence, it's not in any sense guaranteed. Financial instruments (and even stockpiled resources) can be seized, and the eventual outcome of their stockpiling can be much different than intended. In small enough quantities, for short enough timespans, it's reasonable to expect them to be treated like they have always been. But there is an additional level of uncertainty that compounds with interest over time and is likely to increase when passing thresholds that draw broader attention.