This is a special post for quick takes by titotal. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I took a break from engaging with EA topics for like a month or two, and I think it noticeably improved my mental health and productivity, as debating here frequently was actually stressing me out a lot. Which is weird, because the stakes for me posting here are incredibly low: I'm pseudonymous, have no career or personal attachments to EA groups, and I'm highly skeptical that EA efforts will have any noticeable effect on the future of humanity. I can't imagine how stressful these discussions are for people with the opposite positions!
I still have plenty of ideas I want to write up, so I'm not going anywhere, but I'll try to be more considered in where I put my effort.
You are a good and smart commenter, but that is probably generally a sign that you could be doing something more valuable with your time than posting on here. In your case though, that might not actually be true, since you also represent a dissenting perspective that makes things a bit less of an echo chamber on topics like AI safety, and it's possible that does have some marginal influence on what orgs and individuals actually do.
As a pseudonymous poster with a non-EA job who created his account a few months after yours, I've needed to update on the value of prolonged debate on at least meta topics. There was a lot of going in late 2022 / early 2023 where I think having outside voices deeply engaging with debates in the comments was of significant value. I think that is considerably less true for, e.g., the whole Nonlinear controversy in late 2023 / early 2024.
I actually don't really debate on the forums for this very reason. I too am EA-adjacent (yes I'm aware that's a bit of a meme!) and do not work in the EA sphere. I share insights and give feedback, but generally if people disagree I'm happy to leave it at that. I have a very stressful (non-EA) job and so rarely have the bandwidth for something that has no real upside like forum debate. I may make exceptions if someone seems super receptive, but I totally understand why you feel how you do.
I'm a little disheartened at all the downvotes on my last post. I believe an EA public figure used scientifically incorrect language in his public arguments for x-risk, and I put quite a bit of work into explaining why in a good faith and scientifically sourced manner. I'm particularly annoyed that a commenter (with relevant expertise!) was at one point heavily downvoted just for agreeing with me (I saw him at -16 at one point) . Fact checking should take precedence over fandoms.
But... your post was quite inaccurate and strawmanned people extensively?
Eliezer and other commenters compellingly demonstrated this in the comments. I don't think you should get super downvoted, but your post includes a lot of straightforwardly false sentences, so I think the reaction makes sense.
But... your post was quite inaccurate and strawmanned people extensively?
I am legitimately offended by this accusation. I had an organic chemist fact-check the entire thing, and I have included his fact-check into the actual post. Yudkowsky admitted that at least one of his claims was wrong. I explained my exact reasoning for the other problems, and he did not debunk those.
If you can point to any further factual errors in my post after the recent edits, I'm happy to edit those as well.
Yudkowsky's response persuaded me that he didn't intend to say factually incorrect things. (It is also unsourced and full of factual errors, but I don't know if it's worth doing a fact check of a mere comment). But even if he was just badly putting forth an analogy, he is still saying scientifically incorrect things.
I think this sets a pretty terrible precedent about the response to public figures making errors.
Sure. Here are some quotes from the original version of your post:
As a result, I am now certain that the statement “proteins are held together by van der Waals forces rather than covalent bonds” is false. It’s false even if you put hydrogen bonding in the “Van der Waals forces” category, which would be misleading in this context. Nobody who knew about the actual structure of proteins would use the phrase “covalently bonded alternatives to biology”. The entire field of organic chemistry arises from carbon’s thirst for covalent bonds.
This paragraph clearly shows you misunderstood Eliezer. Different proteins are held together almost exclusively by non-covalent forces.
Nobody who knew about the actual structure of proteins would use the phrase “covalently bonded alternatives to biology”.
This is also evidently false, since like dozens of people I know have engaged with Drexlers and Eliezers thoughts on this space, many of which have a pretty deep understanding of chemistry, and would use similar (or the same) phrase. You seem to be invoking some expert consensus that doesn't exist. Indeed, multiple people with PhD level chemistry background have left comments saying they understood Eliezer's point here.
Anybody with a chemistry or biology background who hears someone confidently utter the phrase “covalently bonded equivalents to biology” will immediately have their bullshit alarm triggered, and will probably dismiss everything else you say as well. This also goes for anyone with enough skeptical instinct to google claims to ensure they have the bare minimum of scientific backing.
This is also false. The point makes sense, many people with chemistry or biology background get it, as shown above.
Look, I appreciate the post about the errors in the quantum physics sequence, but you are again vastly overstating the expert consensus here. I have talked with literally 10+ physics PhDs about the quantum physics sequence. Of course there are selection effects, but most people liked it and thought it was great. Yes, it was actually important to add a renormalization term, as you said in your critique, but really none of the points brought up in the sequences depended on it at all.
Like look, when people read your post without actually reading Eliezer's reply, they get the very strong sense that you are claiming that Eliezer is making an error at the level of high-school biology. That somehow he got so confused about chemistry that he didn't understand that a single protein of course is made out of covalent bonds.
But this is really evidently false and kind of absurd. As you can see in a lot of Eliezer's writing, and also his comment level response, Eliezer did not at any point get confused that proteins are made out of covalent bonds. Indeed, to me and Eliezer it seemed so obvious that proteins internally are made out of covalent bonds that I did not consider the possibility that people could somehow interpret this as a claim about the atoms in proteins being held together by Van der Waals forces (how do you even understand what Van der Waals forces are, but don't understand that proteins are internally covalently bonded?). But that misinterpretation seems really what your post was about.
Now, let me look at the most recent version of your post:
Well, it still includes:
As a result, I am now certain that the statement “proteins are held together by van der Waals forces rather than covalent bonds” is false. It’s false even if you put hydrogen bonding in the “Van der Waals forces” category, which would be misleading in this context. Nobody who knew about the actual structure of proteins would use the phrase “covalently bonded alternatives to biology”. The entire field of organic chemistry arises from carbon’s thirst for covalent bonds.
This still seems wrong, though like, you did add some clarifications around it that make it more reasonable.
You did add a whole new section which is quite dense in wrong claims:
> human technology figuring out how to use covalent bonds and metallic bonds, where biology sticks to ionic bonds and proteins held together by van der Waals forces
This is just straight up, explicitly false. Biology does not "stick to ionic bonds and proteins". As I pointed out, biology is made up of covalent bonds at it's very core, and uses them all the time.
Look, this is doubling down on a misinterpretation which at this point you really should have avoided. We are talking about what you call the tertiary structure here. At the level of tertiary structures, and the bonds between proteins, biology does almost solely stick to ionic bonds and proteins held together by Van der Waals forces.
It is the case that sometimes the tertiary structures also use covalent bonds, like in the case of lignin, and I think that's a valid point. It's however not one you had made in your post at all, and just one that Eliezer acknowledges independently. The most recent version of your post now does have a fraction of a sentence, in a quote by your chemistry fact-checker, saying that sometimes tertiary protein structures to use covalent bonds, and I think that's an actual real point that responds to what Eliezer is saying. A post I wouldn't downvote would be one that had that as its main focus, since I think there is a valid critique to be made that biology is in some circumstances capable of using covalent bonds for tertiary structures (as Eliezer acknowledges), but it's not the one you made.
The phrase "covalently bonded equivalents to biology" implicitly states that biology is not covalently bonded. This is false.
Look man, I think you really know by know what Eliezer means by this. Eliezer is talking about alternatives to biology where most of the tertiary structure leverages covalent bonds.
The context of this claim is that Yudkowsky is trying to come up with a new name for deadly Drexler-style nanomachines. He has chosen "covalently bonded bacteria", implying that "covalently bonded bacteria" and normal bacteria are different things. Except that's not true, because bacteria is completely full of covalent bonds.
This is also doubling down on the same misunderstanding. The machinery and tertiary structure of bacteria do not use covalent bonds very much. This is quite different from most current nanomachine designs, which the relevant books hypothesize would be substantially more robust than present biological machinery due to leveraging mostly covalent bonds.
Okay, I just saw this one, but ribosomes are not "general" assemblers, and they cannot replicate "most other products of biology". They do literally one thing, and that is read instructions and link together amino acids to form proteins.
I don't understand what you are talking about here. Basically everything in biology is either made out of proteins or manufactured by proteins. If you can make proteins, you can basically make anything. Proteins are the way most things get done in a cell. The sentence above reads as confused as saying "a CPU is not a general purpose calculator. It does exactly one thing, and that is to read instructions and return the results". Yes, ribosomes read instructions and link together amino acids to form proteins, and that is how biological systems generally assemble things.
For the next two, let's establish the principle that if you say "X is held together by Y instead ofZ", you are implicitly making the statement that "X is not held together by Z", or perhaps that "Z is irrevelant compared to Y when talking about how X is held together", or that "Y is the dominant structural force compared to Z". Otherwise you would not have used the word instead of. Would you utter the phrase "animal bodies are held together by flesh instead of skeletons?"
This one is confused on multiple levels. The meaning of "X is held together by something" of course depends on what level of organization of X you are talking about.
Both of the following sentences are correct:
"Different proteins are held together by Van der Waals and other non-covalent forces instead of covalent bonds". "
A protein is held together by covalent bonds instead of Van der Waals forces and other non-covalent bonds".
Those are fine sentences to say. Yes, it's plausible to get misunderstood, and that's a bit sad, but it doesn't mean you were wrong.
Would you utter the phrase "animal bodies are held together by flesh instead of skeletons?"
This is a random nitpick, but animal bodies are indeed internally held together by flesh instead of skeletons. The skeleton itself is not connected. Bones only provide local structural support against bending and breaking. If I removed your flesh your bones would mostly disconnect and fall into a heap on the ground. Your bones are generally not under much of any strain at any given moment in time, instead you are more like a tensegrity structure where the vast majority of your integrity comes from tension, which comes from your tendons and muscles.
This implicitly makes the statement "proteins are not held together by strong covalent bonds", which is false. Or it could be saying "strong covalent bonds are irrelevant compared to van der waals forces when talking about how proteins are held together", which is also false. edit: Or it is saying that "van der waals forces are the dominant structural force in proteins", which is also false, because this is materially dependent, and some proteins have covalent disulfide links as their dominant structural force.
Even with correction this is still inaccurate. It is correct that non-covalent bonds are the dominant forces for the structure of proteins. Yes, there are some exceptions, like lignin, and that matters, and as I said, I would have upvoted a post talking about that. Yes, it's structurally dependent. But if you aggregate across structures it's true and seems reasonably to describe as being the dominant force.
Hey, I want to thank you for going through the post. I think you've done a good job, and I appreciate it. I'll try to go through and give a similar effort in the replies. Note that I don't want to pressure you to do a re-reply, although you can if you want. I just want to say my piece and defend myself, and I'm happy to let the readers decide from our duelling accounts.
Actually, I will skip ahead first, because I think it illuminates the most where the disagreement lies.
This is a random nitpick, but animal bodies are indeed internally held together by flesh instead of skeletons. The skeleton itself is not connected. Bones only provide local structural support against bending and breaking. If I removed your flesh your bones would mostly disconnect and fall into a heap on the ground. Your bones are generally not under much of any strain at any given moment in time, instead you are more like a tensegrity structure where the vast majority of your integrity comes from tension, which comes from your tendons and muscles.
Yes, if I removed your flesh, your bones would fall to the ground. But similarly, If I removed your skeleton, your flesh would also fall to the ground. Holding a body together is a partnership between bones, muscles and flesh, and if you remove any one, the rest break.
This is kind of the whole of my original point here. Yes, it's perfectly fine to zoom out onto the tertiary structure of a protein, and discuss the make-up of the crosslinks there, if you are clear that's what you're doing. But without the primarily covalent backbone of the system, there is no tertiary structure. So, starting at the beginning:
This paragraph clearly shows you misunderstood Eliezer. Different proteins are held together almost exclusively by non-covalent forces.
I just disagree. Proteins are held together by a combination of covalent bonds and non-covalent forces. If you went in and removed all the covalent bonds, the protein would collapse into nothingness. If you removed all the non-covalent bonds, you would still have that covalent primary structure backbone, which would then snap back into place and reform all the other bonds, rebuilding the protein. (I mean, not every single time, because sometimes undoing denaturation has an energy penalty that is too high). In that sense, it really makes no sense to say that it's held together "almost exclusively by non-covalent forces".
It is true that often non-covalent forces (typically hydrophobic interactions only sometimes Van der waals forces) are the dominant structural force of the 3D structure as a whole. Of course, other times covalent bonds are, as is the case in Keratin-type proteins.
This is also evidently false, since like dozens of people I know have engaged with Drexlers and Eliezers thoughts on this space, many of which have a pretty deep understanding of chemistry, and would use similar (or the same) phrase.
I spent a very, very long time investigating Drexlerian nanotech, and I definitely never saw anything like "covalently bonded equivalents of biology". I think that would be a pretty bad way to describe it, because, as has been established, biology uses covalent bonds at every level. I could see a case for "strictly covalently bonded" though.
You seem to be invoking some expert consensus that doesn't exist. Indeed, multiple people with PhD level chemistry background have left comments saying they understood Eliezer's point here.
I don't want to discount the people who did agree with me or who didn't. I saw a chemist saying they agreed with me and getting downvoted, a protein chemist saying they thought the wording was wrong but they liked the "spaghetti" analogy, and an organic chemist also agreeing with me. Generally the consensus seems to be that the language was badly worded or wrong, but some people found the underlying point defensible or even good. I do agree that some experts are okay with the language.
I think it's worth pointing out that the commentators here and on Lesswrong are disproportionately likely to be Eliezer fans, and be willing to give him the benefit of the doubt. This is not the case for a random person watching a TED talk.
This is also false. The point makes sense, many people with chemistry or biology background get it, as shown above.
You are right that it is an overstatement, I will edit that. However, I maintain that many experts who encounter these badly worded claims will dismiss your argument as a result.
Look, I appreciate the post about the errors in the quantum physics sequence, but you are again vastly overstating the expert consensus here. I have talked with literally 10+ physics PhDs about the quantum physics sequence. Of course there are selection effects, but most people liked it and thought it was great. Yes, it was actually important to add a renormalization term, as you said in your critique, but really none of the points brought up in the sequences depended on it at all.
I think you are underestimating the selection effects here. The physics phd's who thought it sucked are not on lesswrong, they got turned away by the overconfident mistakes. As a physics PHD myself... Eh, it's better than most pop-science stuff, but that's a very low bar. There's plenty more errors in there, and the underlying argument about MWI is pretty bad, but I'll save that for a future post.
Like look, when people read your post without actually reading Eliezer's reply, they get the very strong sense that you are claiming that Eliezer is making an error at the level of high-school biology. That somehow he got so confused about chemistry that he didn't understand that a single protein of course is made out of covalent bonds.
But this is really evidently false and kind of absurd. As you can see in a lot of Eliezer's writing, and also his comment level response, Eliezer did not at any point get confused that proteins are made out of covalent bonds. Indeed, to me and Eliezer it seemed so obvious that proteins internally are made out of covalent bonds that I did not consider the possibility that people could somehow interpret this as a claim about the atoms in proteins being held together by Van der Waals forces (how do you even understand what Van der Waals forces are, but don't understand that proteins are internally covalently bonded?). But that misinterpretation seems really what your post was about.
I don't think it really matters whether he did or did not truly know that the primary structure was covalent. The problem was that at no point, in all of the quotes I found of him discussing the matter, did he clarify that he was talking only about the tertiary structure, or "strictly covalent bonding", or crosslinks between protein folds.
Intentionally or unintentionally, an uninformed listener would get the interpretation: "biology does not use this super duper strong force called "covalent bonds".
Imagine reading those quotes from the perspective of someone who knows nothing about biology, and tell me that that is not the obvious implication of what he says.
I'm happy to give the benefit of the doubt when chatting between friends, but he is using this terminology on podcasts and TED talks. Factual rigour matters.
And, for the record, I don't think it's that unthinkable that he didn't know the primary structure of protein was covalent, it's not that hard of a mistake to make. Unlike say quantum physics, organic chemistry was never a subject I've seen him delve deeply into, and the only source he ever cited on the subject was Drexler.
Look, this is doubling down on a misinterpretation which at this point you really should have avoided. We are talking about what you call the tertiary structure here. At the level of tertiary structures, and the bonds between proteins, biology does almost solely stick to ionic bonds and proteins held together by Van der Waals forces.
Just no. Wait a second, and actually re-read the statement I am responding to. It's a flat statement that "humans utilise covalent bonds, but "biology doesn't". Obviously, biology does "utilise covalent bonds", in that it's made out of covalent bonds. If he only wanted to talk about tertiary structure, he should have said "tertiary structure", and not make a flat statement about all of biology.
Look man, I think you really know by know what Eliezer means by this. Eliezer is talking about alternatives to biology where most of the tertiary structure leverages covalent bonds.
If he means this, he should say that, instead of a different thing that makes no sense. If you say "covalently bonded bacteria", that's the same thing as a regular bacteria.
This is also doubling down on the same misunderstanding. The machinery and tertiary structure of bacteria do not use covalent bonds very much. This is quite different from most current nanomachine designs, which the relevant books hypothesize would be substantially more robust than present biological machinery due to leveraging mostly covalent bonds.
When you say "bacteria", you are talking about the entire bacteria, not just the machinery and tertiary structure. And the entire bacteria does include covalent bonds, or it would fall apart.
Also, they do utilise tertiary covalent bonds for the parts of the cell that need to be very strong.
The relevant books focus on the structural differences: diamondoid nanofactories were hypothesized to be very stable, so that atomically precise placement of molecules could enable new structures that were not available in biology.
I don't understand what you are talking about here. Basically everything in biology is either made out of proteins or manufactured by proteins. If you can make proteins, you can basically make anything. Proteins are the way most things get done in a cell. The sentence above reads as confused as saying "a CPU is not a general purpose calculator. It does exactly one thing, and that is to read instructions and return the results". Yes, ribosomes read instructions and link together amino acids to form proteins, and that is how biological systems generally assemble things.
I see where you're coming from here. It may be a simple matter of bad phrasing on his part leading to a misleading statement. For example, the latter statement:
It should not be very hard for a superintelligence to repurpose ribosomes to build better, more strongly bonded, more energy-dense tiny things that can then have a quite easy time killing everyone
implies that it is the ribosome itself which would directly print out drexlerian nanotech, which of course is impossible. I get that he was probably trying to say the ribosomes will create proteins which in turn create drexlerian nanotech (probably also impossible, but I'll litigate that another day). The phrasing here overstates the ease of this process: uninformed readers will come away thinking "oh, ribosomes are general assemblers, so they can generally assemble nanotech", which is not true.
This one is confused on multiple levels. The meaning of "X is held together by something" of course depends on what level of organization of X you are talking about.
I accept this if you specify the level you are talking about, or it's obvious to both you and your audience which level you are talking about. Neither of these apply to Eliezer's statements.
If you don't specify the level, then the statement is either meaningless, or it's talking about the entire structure. I'm sorry, I am just never going to accept that you can say "instead of covalent bonds", talking generally, when the backbone is covalent bonds.
I think the main takeaway is that you should really avoid using the phrase "held together" altogether, unless it's paired with a more precise descriptor of what structure you are talking about, primarily for all the reasons explored in the post.
Even with correction this is still inaccurate. It is correct that non-covalent bonds are the dominant forces for the structure of proteins. Yes, there are some exceptions, like lignin, and that matters, and as I said, I would have upvoted a post talking about that. Yes, it's structurally dependent. But if you aggregate across structures it's true and seems reasonably to describe as being the dominant force.
He never used the phrase "non-covalent bonds". He only ever said van der waals forces, which is wrong: as multiple people pointed out, if you are forced to state one thing primarily making the 3d structure, it would be hydrophobic bonds.
I think the exceptions are very important, as they negate the whole argument he was making. If biology can make densely covalent bonded things, why doesn't it make all of it's structures covalently bonded? My guess is simply that it's extremely useful to utilize all of the forces available to you, rather than sticking to stiff, rigid, and unflexible structures. But I'll save this for a more detailed research post further down the line.
As a conclusion, I'll just say this: I think that yudkowsky did not sufficiently scientifically vet his arguments before sending them out to the general public. As a result, he said things that gave his audience a misleading picture of the advantages and disadvantages of biology vs nanotech. They were scientifically badly worded, ending up with phrases that were at best misleading and at worst just incorrect. While this whole experience has been pretty exhausting, I hope that it will at least lead to improved scientific terminology in the future.
I will eventually write a more detailed analysis of biology vs Drexler, and you can better believe that one will be extensively fact checked before it goes out.
I will do another round of edits tomorrow, and I'll probably let that be it, there is only so much time in the world one can devote to posting.
It does look like there is an interpretation of EYs basic claims which is roughly reasonable and one which is clearly false and unreasonable, and you assumed he meant the clearly unreasonable thing and attack that.
I think absent further evidence, it's fair for others to say "he couldn't have possibly meant that" and move on.
That's fair enough and levels of Background understanding vary (I don't have a relevant PhD either), but then the criticism should be about this point being easily misunderstood rather than making a big deal about the strawman position being factually wrong. In which case it would also be much more constructive than adversarial criticism.
I think part of titotal’s point is it’s not the ‘strawman’ interpretation but the straightforward one, and having it framed that way would understandably be frustrating. It sounds like he also disagrees with Eliezer’s actual, badly communicated argument [edit: about the size of potential improvements on biology] anyway though? Based on the response to Habryka
Yeah, I think it would have been much better for him to say "proteins are shaped by..." rather than "proteins are held together by...", and to give some context for what that means. Seems fair to criticize his communication. But the quotes and examples in the linked post are more consistent with him understanding that and wording it poorly, or assuming too much of his audience, rather than him not understanding that proteins use covalent bonds.
The selected quotes do give me the impression Eliezer is underestimating what nature can accomplish relative to design, but I haven't read any of them in context so that doesn't prove much.
I took a break from engaging with EA topics for like a month or two, and I think it noticeably improved my mental health and productivity, as debating here frequently was actually stressing me out a lot. Which is weird, because the stakes for me posting here are incredibly low: I'm pseudonymous, have no career or personal attachments to EA groups, and I'm highly skeptical that EA efforts will have any noticeable effect on the future of humanity. I can't imagine how stressful these discussions are for people with the opposite positions!
I still have plenty of ideas I want to write up, so I'm not going anywhere, but I'll try to be more considered in where I put my effort.
You are a good and smart commenter, but that is probably generally a sign that you could be doing something more valuable with your time than posting on here. In your case though, that might not actually be true, since you also represent a dissenting perspective that makes things a bit less of an echo chamber on topics like AI safety, and it's possible that does have some marginal influence on what orgs and individuals actually do.
As a pseudonymous poster with a non-EA job who created his account a few months after yours, I've needed to update on the value of prolonged debate on at least meta topics. There was a lot of going in late 2022 / early 2023 where I think having outside voices deeply engaging with debates in the comments was of significant value. I think that is considerably less true for, e.g., the whole Nonlinear controversy in late 2023 / early 2024.
Debating still takes time and energy which reduces the time and energy available elsewhere.
I actually don't really debate on the forums for this very reason. I too am EA-adjacent (yes I'm aware that's a bit of a meme!) and do not work in the EA sphere. I share insights and give feedback, but generally if people disagree I'm happy to leave it at that. I have a very stressful (non-EA) job and so rarely have the bandwidth for something that has no real upside like forum debate. I may make exceptions if someone seems super receptive, but I totally understand why you feel how you do.
I'm a little disheartened at all the downvotes on my last post. I believe an EA public figure used scientifically incorrect language in his public arguments for x-risk, and I put quite a bit of work into explaining why in a good faith and scientifically sourced manner. I'm particularly annoyed that a commenter (with relevant expertise!) was at one point heavily downvoted just for agreeing with me (I saw him at -16 at one point) . Fact checking should take precedence over fandoms.
But... your post was quite inaccurate and strawmanned people extensively?
Eliezer and other commenters compellingly demonstrated this in the comments. I don't think you should get super downvoted, but your post includes a lot of straightforwardly false sentences, so I think the reaction makes sense.
I am legitimately offended by this accusation. I had an organic chemist fact-check the entire thing, and I have included his fact-check into the actual post. Yudkowsky admitted that at least one of his claims was wrong. I explained my exact reasoning for the other problems, and he did not debunk those.
If you can point to any further factual errors in my post after the recent edits, I'm happy to edit those as well.
Yudkowsky's response persuaded me that he didn't intend to say factually incorrect things. (It is also unsourced and full of factual errors, but I don't know if it's worth doing a fact check of a mere comment). But even if he was just badly putting forth an analogy, he is still saying scientifically incorrect things.
I think this sets a pretty terrible precedent about the response to public figures making errors.
Sure. Here are some quotes from the original version of your post:
This paragraph clearly shows you misunderstood Eliezer. Different proteins are held together almost exclusively by non-covalent forces.
This is also evidently false, since like dozens of people I know have engaged with Drexlers and Eliezers thoughts on this space, many of which have a pretty deep understanding of chemistry, and would use similar (or the same) phrase. You seem to be invoking some expert consensus that doesn't exist. Indeed, multiple people with PhD level chemistry background have left comments saying they understood Eliezer's point here.
This is also false. The point makes sense, many people with chemistry or biology background get it, as shown above.
Look, I appreciate the post about the errors in the quantum physics sequence, but you are again vastly overstating the expert consensus here. I have talked with literally 10+ physics PhDs about the quantum physics sequence. Of course there are selection effects, but most people liked it and thought it was great. Yes, it was actually important to add a renormalization term, as you said in your critique, but really none of the points brought up in the sequences depended on it at all.
Like look, when people read your post without actually reading Eliezer's reply, they get the very strong sense that you are claiming that Eliezer is making an error at the level of high-school biology. That somehow he got so confused about chemistry that he didn't understand that a single protein of course is made out of covalent bonds.
But this is really evidently false and kind of absurd. As you can see in a lot of Eliezer's writing, and also his comment level response, Eliezer did not at any point get confused that proteins are made out of covalent bonds. Indeed, to me and Eliezer it seemed so obvious that proteins internally are made out of covalent bonds that I did not consider the possibility that people could somehow interpret this as a claim about the atoms in proteins being held together by Van der Waals forces (how do you even understand what Van der Waals forces are, but don't understand that proteins are internally covalently bonded?). But that misinterpretation seems really what your post was about.
Now, let me look at the most recent version of your post:
Well, it still includes:
This still seems wrong, though like, you did add some clarifications around it that make it more reasonable.
You did add a whole new section which is quite dense in wrong claims:
Look, this is doubling down on a misinterpretation which at this point you really should have avoided. We are talking about what you call the tertiary structure here. At the level of tertiary structures, and the bonds between proteins, biology does almost solely stick to ionic bonds and proteins held together by Van der Waals forces.
It is the case that sometimes the tertiary structures also use covalent bonds, like in the case of lignin, and I think that's a valid point. It's however not one you had made in your post at all, and just one that Eliezer acknowledges independently. The most recent version of your post now does have a fraction of a sentence, in a quote by your chemistry fact-checker, saying that sometimes tertiary protein structures to use covalent bonds, and I think that's an actual real point that responds to what Eliezer is saying. A post I wouldn't downvote would be one that had that as its main focus, since I think there is a valid critique to be made that biology is in some circumstances capable of using covalent bonds for tertiary structures (as Eliezer acknowledges), but it's not the one you made.
Look man, I think you really know by know what Eliezer means by this. Eliezer is talking about alternatives to biology where most of the tertiary structure leverages covalent bonds.
This is also doubling down on the same misunderstanding. The machinery and tertiary structure of bacteria do not use covalent bonds very much. This is quite different from most current nanomachine designs, which the relevant books hypothesize would be substantially more robust than present biological machinery due to leveraging mostly covalent bonds.
I don't understand what you are talking about here. Basically everything in biology is either made out of proteins or manufactured by proteins. If you can make proteins, you can basically make anything. Proteins are the way most things get done in a cell. The sentence above reads as confused as saying "a CPU is not a general purpose calculator. It does exactly one thing, and that is to read instructions and return the results". Yes, ribosomes read instructions and link together amino acids to form proteins, and that is how biological systems generally assemble things.
This one is confused on multiple levels. The meaning of "X is held together by something" of course depends on what level of organization of X you are talking about.
Both of the following sentences are correct:
Those are fine sentences to say. Yes, it's plausible to get misunderstood, and that's a bit sad, but it doesn't mean you were wrong.
This is a random nitpick, but animal bodies are indeed internally held together by flesh instead of skeletons. The skeleton itself is not connected. Bones only provide local structural support against bending and breaking. If I removed your flesh your bones would mostly disconnect and fall into a heap on the ground. Your bones are generally not under much of any strain at any given moment in time, instead you are more like a tensegrity structure where the vast majority of your integrity comes from tension, which comes from your tendons and muscles.
Even with correction this is still inaccurate. It is correct that non-covalent bonds are the dominant forces for the structure of proteins. Yes, there are some exceptions, like lignin, and that matters, and as I said, I would have upvoted a post talking about that. Yes, it's structurally dependent. But if you aggregate across structures it's true and seems reasonably to describe as being the dominant force.
Hey, I want to thank you for going through the post. I think you've done a good job, and I appreciate it. I'll try to go through and give a similar effort in the replies. Note that I don't want to pressure you to do a re-reply, although you can if you want. I just want to say my piece and defend myself, and I'm happy to let the readers decide from our duelling accounts.
Actually, I will skip ahead first, because I think it illuminates the most where the disagreement lies.
Yes, if I removed your flesh, your bones would fall to the ground. But similarly, If I removed your skeleton, your flesh would also fall to the ground. Holding a body together is a partnership between bones, muscles and flesh, and if you remove any one, the rest break.
This is kind of the whole of my original point here. Yes, it's perfectly fine to zoom out onto the tertiary structure of a protein, and discuss the make-up of the crosslinks there, if you are clear that's what you're doing. But without the primarily covalent backbone of the system, there is no tertiary structure. So, starting at the beginning:
I just disagree. Proteins are held together by a combination of covalent bonds and non-covalent forces. If you went in and removed all the covalent bonds, the protein would collapse into nothingness. If you removed all the non-covalent bonds, you would still have that covalent primary structure backbone, which would then snap back into place and reform all the other bonds, rebuilding the protein. (I mean, not every single time, because sometimes undoing denaturation has an energy penalty that is too high). In that sense, it really makes no sense to say that it's held together "almost exclusively by non-covalent forces".
It is true that often non-covalent forces (typically hydrophobic interactions only sometimes Van der waals forces) are the dominant structural force of the 3D structure as a whole. Of course, other times covalent bonds are, as is the case in Keratin-type proteins.
I spent a very, very long time investigating Drexlerian nanotech, and I definitely never saw anything like "covalently bonded equivalents of biology". I think that would be a pretty bad way to describe it, because, as has been established, biology uses covalent bonds at every level. I could see a case for "strictly covalently bonded" though.
I don't want to discount the people who did agree with me or who didn't. I saw a chemist saying they agreed with me and getting downvoted, a protein chemist saying they thought the wording was wrong but they liked the "spaghetti" analogy, and an organic chemist also agreeing with me. Generally the consensus seems to be that the language was badly worded or wrong, but some people found the underlying point defensible or even good. I do agree that some experts are okay with the language.
I think it's worth pointing out that the commentators here and on Lesswrong are disproportionately likely to be Eliezer fans, and be willing to give him the benefit of the doubt. This is not the case for a random person watching a TED talk.
You are right that it is an overstatement, I will edit that. However, I maintain that many experts who encounter these badly worded claims will dismiss your argument as a result.
I think you are underestimating the selection effects here. The physics phd's who thought it sucked are not on lesswrong, they got turned away by the overconfident mistakes. As a physics PHD myself... Eh, it's better than most pop-science stuff, but that's a very low bar. There's plenty more errors in there, and the underlying argument about MWI is pretty bad, but I'll save that for a future post.
I don't think it really matters whether he did or did not truly know that the primary structure was covalent. The problem was that at no point, in all of the quotes I found of him discussing the matter, did he clarify that he was talking only about the tertiary structure, or "strictly covalent bonding", or crosslinks between protein folds.
Intentionally or unintentionally, an uninformed listener would get the interpretation: "biology does not use this super duper strong force called "covalent bonds".
Imagine reading those quotes from the perspective of someone who knows nothing about biology, and tell me that that is not the obvious implication of what he says.
I'm happy to give the benefit of the doubt when chatting between friends, but he is using this terminology on podcasts and TED talks. Factual rigour matters.
And, for the record, I don't think it's that unthinkable that he didn't know the primary structure of protein was covalent, it's not that hard of a mistake to make. Unlike say quantum physics, organic chemistry was never a subject I've seen him delve deeply into, and the only source he ever cited on the subject was Drexler.
Just no. Wait a second, and actually re-read the statement I am responding to. It's a flat statement that "humans utilise covalent bonds, but "biology doesn't". Obviously, biology does "utilise covalent bonds", in that it's made out of covalent bonds. If he only wanted to talk about tertiary structure, he should have said "tertiary structure", and not make a flat statement about all of biology.
If he means this, he should say that, instead of a different thing that makes no sense. If you say "covalently bonded bacteria", that's the same thing as a regular bacteria.
When you say "bacteria", you are talking about the entire bacteria, not just the machinery and tertiary structure. And the entire bacteria does include covalent bonds, or it would fall apart.
Also, they do utilise tertiary covalent bonds for the parts of the cell that need to be very strong.
The relevant books focus on the structural differences: diamondoid nanofactories were hypothesized to be very stable, so that atomically precise placement of molecules could enable new structures that were not available in biology.
I see where you're coming from here. It may be a simple matter of bad phrasing on his part leading to a misleading statement. For example, the latter statement:
implies that it is the ribosome itself which would directly print out drexlerian nanotech, which of course is impossible. I get that he was probably trying to say the ribosomes will create proteins which in turn create drexlerian nanotech (probably also impossible, but I'll litigate that another day). The phrasing here overstates the ease of this process: uninformed readers will come away thinking "oh, ribosomes are general assemblers, so they can generally assemble nanotech", which is not true.
I accept this if you specify the level you are talking about, or it's obvious to both you and your audience which level you are talking about. Neither of these apply to Eliezer's statements.
If you don't specify the level, then the statement is either meaningless, or it's talking about the entire structure. I'm sorry, I am just never going to accept that you can say "instead of covalent bonds", talking generally, when the backbone is covalent bonds.
I think the main takeaway is that you should really avoid using the phrase "held together" altogether, unless it's paired with a more precise descriptor of what structure you are talking about, primarily for all the reasons explored in the post.
He never used the phrase "non-covalent bonds". He only ever said van der waals forces, which is wrong: as multiple people pointed out, if you are forced to state one thing primarily making the 3d structure, it would be hydrophobic bonds.
I think the exceptions are very important, as they negate the whole argument he was making. If biology can make densely covalent bonded things, why doesn't it make all of it's structures covalently bonded? My guess is simply that it's extremely useful to utilize all of the forces available to you, rather than sticking to stiff, rigid, and unflexible structures. But I'll save this for a more detailed research post further down the line.
As a conclusion, I'll just say this: I think that yudkowsky did not sufficiently scientifically vet his arguments before sending them out to the general public. As a result, he said things that gave his audience a misleading picture of the advantages and disadvantages of biology vs nanotech. They were scientifically badly worded, ending up with phrases that were at best misleading and at worst just incorrect. While this whole experience has been pretty exhausting, I hope that it will at least lead to improved scientific terminology in the future.
I will eventually write a more detailed analysis of biology vs Drexler, and you can better believe that one will be extensively fact checked before it goes out.
I will do another round of edits tomorrow, and I'll probably let that be it, there is only so much time in the world one can devote to posting.
It does look like there is an interpretation of EYs basic claims which is roughly reasonable and one which is clearly false and unreasonable, and you assumed he meant the clearly unreasonable thing and attack that. I think absent further evidence, it's fair for others to say "he couldn't have possibly meant that" and move on.
As someone in the ‘general public rather than chemistry/physics PhD’ group, which Eliezer is saying he’s targeting, I definitely thought he meant that
That's fair enough and levels of Background understanding vary (I don't have a relevant PhD either), but then the criticism should be about this point being easily misunderstood rather than making a big deal about the strawman position being factually wrong. In which case it would also be much more constructive than adversarial criticism.
I think part of titotal’s point is it’s not the ‘strawman’ interpretation but the straightforward one, and having it framed that way would understandably be frustrating. It sounds like he also disagrees with Eliezer’s actual, badly communicated argument [edit: about the size of potential improvements on biology] anyway though? Based on the response to Habryka
Yeah, I think it would have been much better for him to say "proteins are shaped by..." rather than "proteins are held together by...", and to give some context for what that means. Seems fair to criticize his communication. But the quotes and examples in the linked post are more consistent with him understanding that and wording it poorly, or assuming too much of his audience, rather than him not understanding that proteins use covalent bonds.
The selected quotes do give me the impression Eliezer is underestimating what nature can accomplish relative to design, but I haven't read any of them in context so that doesn't prove much.
Yeah I can empathise, it's hard isn't it. Though I am glad people get to freely vote. I'm sad for you.