R

Roko

-225 karmaJoined

Posts
7

Sorted by New

Comments
21

If you mean some parts of technical alignment, then perhaps that's true, but I mean alignment in the broad sense of creating good outcomes.

> AI researchers are more in the former category

Yeah but people in the former category don't matter much in terms of outcomes. Making a better sparse autoencoder won't change the world at the margin, just like technocrats working to make soviet central planning better ultimately didn't change the world because they were making incremental progress in the wrong direction. 

Unfortunately I think that a subject like bacteriology is more resistant to bad epistenics than something like AI alignment or effective altruism. 

And I think this critique just sort of generalizes to a fairly general critique of EA: if you want to make a widget that's 5% better, you can specialize in widget making and then go home and believe in crystal healing and diversity and inclusion after work. 

But if you want to make impactful changes to the world and you believe in crystal healing and so on, you will probably be drawn away from correct strategies because correct strategies for improving the world tend to require an accurate world model including being accurate about things that are controversial. 

Communism is perhaps the prime example of this from the 20th century: many people seriously believed that communism was good, and they believed that so much that they rejected evidence to the contrary. Entire continents have been ravaged as a result. 

HBD denial (race communism) is the communism of the present. 

Would you trust an AI alignment researcher who supported Lysenkoism in the era when it was popular in the soviet union? 

it's more like maybe HBD and wokeism are two sides of a toxic dynamic where it would be better if we could get back to other concerns.

 

Unfortunately this "toxic dynamic" is also known as truth-based versus consequences-based epistemology and that is a dynamic that you absolutely cannot escape because the ability to alter social consensus beliefs for the benefits of special interests is just a generic problem. It will also pop up in the AI debate. 

If someone is interested in (3), they'll hopefully understand that a lot of things that are pressing problems today will either no longer matter in 1-20 years because we're all dead, or they'll be more easily solvable with the help of aligned powerful AIs 

 

Actually I think that people's thinking about AI has become somewhat broken and I'm starting to see the same dark-side epistemology that gave us HBD-denialism seep into the AI community. 

But, take a step back. 

Suppose you have a community of biologists who all believe in Lysenkoism. Then, despite their repeated failures to improve crop yields, the country is bailed out by a large external food source. 

Would you be willing to overlook their belief in Lysenkoism and have these people start working on cancer biology, aging and other areas? 

Or, look at another example. You ask a self-professed mathematician whether he thinks that all continuous functions are differentiable. He says they are, and it's so obvious that it requires no proof. Do you trust this mathematician to generally provide good advice? 

My point is that process matters in science and epistemology. You can't sweep the bad process of creationists under the carpet and expect them to continue to produce good results on other issues. Their process is broken. 

How is HBD action-relevant for EA in a pre-AGI world? 

 

I don't think it is, because I think AI will replace humans in all economic roles within 5-15 years. But I think the same dark-side intellectual tactics that gave rise to HBD-denialism will contaminate our thinking about AI, just in different ways.  

Load more