*When I click on the "listen online" option it seems broken (using this on a mac)
*When I click on the "AGI safety fundamentals" courses as podcasts, they take me to the "EA forum curated and popular" podcast. Not sure if this is intentional, or if they're meant to point to a podcast containing just the course
Fair enough. I admit that I skimmed the post quickly, for which I apologize, and part of this was certainly a knee-jerk reaction to even considering Leverage as a serious intellectual project rather than a total failure as such, which is not entirely fair. But I think maybe a version of this post I would significantly prefer would first explain your interest in Leverage specifically: that while they are a particularly egregious failure of the closed-research genre, it's interesting to understand exactly how they failed and how the idea of a fast, less-than-fully transparent think tank can be salvaged. It does bother me that you don't try to look for other examples of organizations that do some part of this more effectively, and I have trouble believing that they don't exist. It reads a bit like an analysis of nation-building that focuses specifically on the mistakes and complexities of North Korea without trying to compare it to other less awful entities.
Edit: I mostly retract this comment. I skimmed and didn't read the post carefully (something one should never do before leaving a negative comment) and interpreted it as "Leverage wasn't perfect, but it is worth trying to make Leverage 2.0 work or have similar projects with small changes". On rereading, I see that Jeff's emphasis is more on analyzing and quantifying the failure modes than on salvaging the idea.
That said, I just want to point out that (at least as far as I understand it), there is a significant collection of people within and around EA who think that Leverage is a uniquely awful organization which suffered a multilevel failure extremely reminiscent of your run-of-the mill cult (not just for those who left it, but also for many people who are still in it), which soft-core threatens members to avoid negative publicity, exerts psychological control on members in ways that seem scary and evil. This is context that I think some people reading the sterilized publicity around Leverage will lack.
There are many directions from which people could approach Leverage 1.0, but the one that I'm most interested in is lessons for people considering attempting similar things in the future.
I think there's a really clear lesson here: don't.
I'll elaborate: Leverage was a multilevel failure. A fundamentally dishonest and charismatic leader. A group of people very convinced that their particular chain of flimsy inferences led them to some higher truth that gave them advantages over everyone else. A frenzied sense of secrecy and importance. Ultimately, psychological harm and abuse.
It is very clearly a negative example, and if someone is genuinely trying to gain some positive insight into a project from "things they did right" (or noticeably imitate techniques from that project), that would make me significantly less likely to think of them as being on the right track.
There are examples of better "secret projects" - the Manhattan project as well as other high-security government organizations, various secret revolutionary groups like the early US revolutionaries, the abolitionist movement and the underground railroad, even various pro-social masonic orders. Having as one's go-to example of something to emulate an organization that significantly crossed the line into cult territory (or at least into Aleister Crowley level grandiosity around a bad actor) would indicate to me a potential enlarged sense of self-importance, an emphasis on deference and exclusivity ("being on our team") instead of competence and accountability, and a lack of emphasis on appropriate levels of humility and self-regulation.
To be clear, I believe in decoupling and don't think it's wrong to learn from bad actors. But with such a deeply rotten track record, and so many decent organizations that are better than it along all parameters, Leverage is perhaps the clearest example of a situation where people should just "say oops" and stop looking for ways to gain any value from it (other than as a cautionary tale) that I have heard of in the EA/LW community.
I'm really curious about your mention of the "schism" pattern because I both haven't seen it and I sort of believe a version of it. What were the schism posts? And why are they bad?
I don't know if what you call "schismatics" want to burn the commons of EA cooperation (which would be bad), or if they just want to stop the tendency in EA (and really, everywhere) of people pushing for everyone to adopt convergent views (the focus of "if you believe X you should also believe Y" which I see and dislike in EA, versus "I don't think X is the most important thing, but if you believe X here are some ways you could can do it more effectively" which I would like to see more).
Though I can see myself changing my mind on this, I currently like the idea of a more loose EA community with more moving parts that has a larger spectrum of vaguely positive-EV views. I've actually considered writing something about it inspired by this post by Eric Neyman https://ericneyman.wordpress.com/2021/06/05/social-behavior-curves-equilibria-and-radicalism/ which quantifies, among other things, the intuition that people are more likely to change their mind/behavior in a significant way if there is a larger spectrum of points of view rather than a more bimodal distribution.
I like hypothesis generation, and I particularly like that in this post a few of the points are mutually exclusive (like numbers 7 and 10), which should happen in a hypothesis generation post. However this list, as well as the topic, feels lazy to me, in the sense of needing much more specificity in other to generate more light than heat.
I think my main issue is the extremely vague use of"quality" here. It's ok to use vague terms when a concept is hard to define, but in this case it feels like there are more useful ways to narrow it down. For example you could say "the average post seems less informative/well-researched" or "the average poster seems less experienced/ qualified", or "I learned more from the old forum than the new one" (I think especially a focus on your experience would make the issue more precise, and open up new options such as "posts became less fun once I learned all the basics and new people who are just learning them became less interesting to me"). I would like to see a hypothesis generation post that focuses much more on the specific ways that posts are "worse" (and generates hypotheses on what they are) rather than on reasons for this to be the case. I suspect that once a concrete question is asked, the potential reasons will become more concrete and testable.
Another issue is that I think a lot of the points are more properly "reasons that posts on a forum can be bad" rather than issues with current vs old posts and I have trouble believing that these issues were absent or better in the past. This would also be solved by trying to make the complaint specific.
I'm one of the people who submitted a post right before the deadline of the criticism contest. FWIW I think number 6 is off base. In my case, the deadline felt like a Schelling point. My post was long and kind of technical, and I didn't have any expectation of getting money - though having a fake deadline was very helpful and I would probably not have written it without the contest. I don't think that any of the posts that got prizes were written with an expectation of making a profit. They all looked like an investment of multiple hours by talented people who could have made much more money (at least in expectation) by doing something else. In order for someone profit-motivated to take advantage of this they would have to be involved and knowledgeable enough to write a competitive post, and unable to make money in essentially any other way. This seems like an unlikely combination, but if it does exist then I'd assume that supporting such people financially is an additional benefit rather than a problem.
Thanks! I didn't fully understand what people meant by that and how it's related to various forms of longtermism. Skimming the linked post was helpful to get a better picture.
Some possible bugs:
*When I click on the "listen online" option it seems broken (using this on a mac)
*When I click on the "AGI safety fundamentals" courses as podcasts, they take me to the "EA forum curated and popular" podcast. Not sure if this is intentional, or if they're meant to point to a podcast containing just the course
Thanks! But I see your point
Certainly not deliberately. I'll try to read it more carefully and update my comment
Fair enough. I admit that I skimmed the post quickly, for which I apologize, and part of this was certainly a knee-jerk reaction to even considering Leverage as a serious intellectual project rather than a total failure as such, which is not entirely fair. But I think maybe a version of this post I would significantly prefer would first explain your interest in Leverage specifically: that while they are a particularly egregious failure of the closed-research genre, it's interesting to understand exactly how they failed and how the idea of a fast, less-than-fully transparent think tank can be salvaged. It does bother me that you don't try to look for other examples of organizations that do some part of this more effectively, and I have trouble believing that they don't exist. It reads a bit like an analysis of nation-building that focuses specifically on the mistakes and complexities of North Korea without trying to compare it to other less awful entities.
Edit: I mostly retract this comment. I skimmed and didn't read the post carefully (something one should never do before leaving a negative comment) and interpreted it as "Leverage wasn't perfect, but it is worth trying to make Leverage 2.0 work or have similar projects with small changes". On rereading, I see that Jeff's emphasis is more on analyzing and quantifying the failure modes than on salvaging the idea.
That said, I just want to point out that (at least as far as I understand it), there is a significant collection of people within and around EA who think that Leverage is a uniquely awful organization which suffered a multilevel failure extremely reminiscent of your run-of-the mill cult (not just for those who left it, but also for many people who are still in it), which soft-core threatens members to avoid negative publicity, exerts psychological control on members in ways that seem scary and evil. This is context that I think some people reading the sterilized publicity around Leverage will lack.
I think there's a really clear lesson here: don't.I'll elaborate: Leverage was a multilevel failure. A fundamentally dishonest and charismatic leader. A group of people very convinced that their particular chain of flimsy inferences led them to some higher truth that gave them advantages over everyone else. A frenzied sense of secrecy and importance. Ultimately, psychological harm and abuse.It is very clearly a negative example, and if someone is genuinely trying to gain some positive insight into a project from "things they did right" (or noticeably imitate techniques from that project), that would make me significantly less likely to think of them as being on the right track.There are examples of better "secret projects" - the Manhattan project as well as other high-security government organizations, various secret revolutionary groups like the early US revolutionaries, the abolitionist movement and the underground railroad, even various pro-social masonic orders. Having as one's go-to example of something to emulate an organization that significantly crossed the line into cult territory (or at least into Aleister Crowley level grandiosity around a bad actor) would indicate to me a potential enlarged sense of self-importance, an emphasis on deference and exclusivity ("being on our team") instead of competence and accountability, and a lack of emphasis on appropriate levels of humility and self-regulation.To be clear, I believe in decoupling and don't think it's wrong to learn from bad actors. But with such a deeply rotten track record, and so many decent organizations that are better than it along all parameters, Leverage is perhaps the clearest example of a situation where people should just "say oops" and stop looking for ways to gain any value from it (other than as a cautionary tale) that I have heard of in the EA/LW community.I enjoyed this post a lot!
I'm really curious about your mention of the "schism" pattern because I both haven't seen it and I sort of believe a version of it. What were the schism posts? And why are they bad?
I don't know if what you call "schismatics" want to burn the commons of EA cooperation (which would be bad), or if they just want to stop the tendency in EA (and really, everywhere) of people pushing for everyone to adopt convergent views (the focus of "if you believe X you should also believe Y" which I see and dislike in EA, versus "I don't think X is the most important thing, but if you believe X here are some ways you could can do it more effectively" which I would like to see more).
Though I can see myself changing my mind on this, I currently like the idea of a more loose EA community with more moving parts that has a larger spectrum of vaguely positive-EV views. I've actually considered writing something about it inspired by this post by Eric Neyman https://ericneyman.wordpress.com/2021/06/05/social-behavior-curves-equilibria-and-radicalism/ which quantifies, among other things, the intuition that people are more likely to change their mind/behavior in a significant way if there is a larger spectrum of points of view rather than a more bimodal distribution.
I like hypothesis generation, and I particularly like that in this post a few of the points are mutually exclusive (like numbers 7 and 10), which should happen in a hypothesis generation post. However this list, as well as the topic, feels lazy to me, in the sense of needing much more specificity in other to generate more light than heat.
I think my main issue is the extremely vague use of"quality" here. It's ok to use vague terms when a concept is hard to define, but in this case it feels like there are more useful ways to narrow it down. For example you could say "the average post seems less informative/well-researched" or "the average poster seems less experienced/ qualified", or "I learned more from the old forum than the new one" (I think especially a focus on your experience would make the issue more precise, and open up new options such as "posts became less fun once I learned all the basics and new people who are just learning them became less interesting to me"). I would like to see a hypothesis generation post that focuses much more on the specific ways that posts are "worse" (and generates hypotheses on what they are) rather than on reasons for this to be the case. I suspect that once a concrete question is asked, the potential reasons will become more concrete and testable.
Another issue is that I think a lot of the points are more properly "reasons that posts on a forum can be bad" rather than issues with current vs old posts and I have trouble believing that these issues were absent or better in the past. This would also be solved by trying to make the complaint specific.
I'm one of the people who submitted a post right before the deadline of the criticism contest. FWIW I think number 6 is off base. In my case, the deadline felt like a Schelling point. My post was long and kind of technical, and I didn't have any expectation of getting money - though having a fake deadline was very helpful and I would probably not have written it without the contest. I don't think that any of the posts that got prizes were written with an expectation of making a profit. They all looked like an investment of multiple hours by talented people who could have made much more money (at least in expectation) by doing something else. In order for someone profit-motivated to take advantage of this they would have to be involved and knowledgeable enough to write a competitive post, and unable to make money in essentially any other way. This seems like an unlikely combination, but if it does exist then I'd assume that supporting such people financially is an additional benefit rather than a problem.
But I agree with your meta-point that I implicitly assumed SSA together with my "assumption 5" and SSA might not follow from the other assumptions
Thanks! I didn't fully understand what people meant by that and how it's related to various forms of longtermism. Skimming the linked post was helpful to get a better picture.