Building Cooperative Epistemology (Response to "EA has a Lying Problem", among other things)

I strongly agree with the points Ben Hoffman has been making (mostly in the other threads) about the epistemic problems caused by holding criticism to a higher standard than praise. I also think that we should be fairly mindful that providing public criticism can have a high social cost to the person making the criticism, even though they are providing a public service.

There are definitely ways that Sarah could have improved her post. But that is basically always going to be true of any blog post unless one spends 20+ hours writing it.

I personally have a number of criticisms of EA (despite overall being a strong proponent of the movement) that I am fairly unlikely to share publicly, due to the following dynamic: anything I write that wouldn't incur unacceptably high social costs would have to be a highly watered-down version of the original point, and/or involve so much of my time to write carefully that it wouldn't be worthwhile.

While I'm sympathetic to the fact that there's also a lot of low-quality / lazy criticism of EA, I don't think responses that involve setting a high bar for high-quality criticism are the right way to go.

(Note that I don't think that EA is worse than is typical in terms of accepting criticism, though I do think that there are other groups / organizations that substantially outperform EA, which provides an existence proof that one can do much better.)

Issue 1:

The title and tone of this post is playing with fire, i.e courting controversy, in a way that (I think, but am not sure) undermines its goals.

A, there's the fact that describing these as "lying" seems approximately as true as the first two claims, which other people have mentioned. In a post about holding ourselves to high standards, this is kind of a big deal. Others have mentioned this.

B: Personal integrity/honesty is only one element you need to have a good epistemic culture. Other elements you need include trust, and respect for people's time, attention, and emotions.

Just as every decision to bend the truth has consequences, every decision to inflame emotions has consequences, and these can be just as damaging.

I assume (hope) it was a deliberate choice to use a provocative title that'd grab attention. I think part of the goal was to punish the EA Establishment for not responding well to criticism and attempting to control said criticism.

That may not be a bad choice. Maybe it's necessary but it's a questionable one.

The default world (see: modern politics, and news) is a race to the bottom of outrage and manufactured controversy. People love controversy. I love controversy. I felt an urge to share this article on facebook and say things off the cuff about it. I resisted, because I think it would be harmful to the epistemic integrity of EA.

Maybe it's necessary to write a provocative title with a hazy definition of "lying" in order to get everyone's attention and force a conversation. (In the same way it may be necessary to exaggerate global warming by 4x to get Jane Q Public to care). But it is certainly not the platonic ideal of the epistemic culture we need to build.

Hi everyone! I’m here to formally respond to Sarah’s article, on behalf of ACE. It’s difficult to determine where the response should go, as it seems there are many discussions, and reposting appears to be discouraged. I’ve decided to post here on the EA forum (as it tends to be the central meeting place for EAs), and will try to direct people from other places to this longer response.

Firstly, I’d like to clarify why we have not inserted ourselves into the discussion happening in multiple Facebook groups and fora. We have recently implemented a formal social media policy which encourages ACE staff to respond to comments about our work with great consideration, and in a way that accurately reflects our views (as opposed to those of one staff member). We are aware that this might come across as “radio silence” or lack of concern for the criticism at hand—but that is not the case. Whenever there are legitimate critiques about our work, we take it very seriously. When there are accusations of intent to deceive, we do not take them lightly. The last thing we want to do is respond in haste only to realize that we had not given the criticism enough consideration. We also want to allow the community to discuss amongst themselves prior to posting a response. This is not only to encourage discussion amongst individual members of the community, but also so that we can prioritize responding to the concerns shared by the greatest number of community members.

It is clear to us now that we have failed to adequately communicate the uncertainty surrounding the outcomes of our leafleting intervention report. We absolutely disagree with claims of intentional deception and the characterization of our staff as acting in bad-faith—we have never tried to hide our uncertainty about the existing leafleting research report, and as others have pointed out, it is clearly stated throughout the site where leafleting is mentioned. However, our reasoning that these disclaimers would be obvious was based on the assumption that those interested in the report would read it in its entirety. After reading the responses to this article, it’s obvious that we have not made these disclaimers as apparent as they should be. We have added a longer disclaimer to the top of our leafleting report page, expressing our current thoughts and noting that we will update the report sometime in 2017.

In addition, we have decided to remove the impact calculator (a tool which included an ability to enter donations directed to leafleting and receive estimates of high and low bounds of animals spared) from our website entirely until we feel more confident that it is not misleading to those unfamiliar with cost effectiveness calculations and/or an understanding of how the low/best/high error bounds exemplify the uncertainty regarding those numbers. It is not typical for us to remove content from the site, but we intend to operate with abundant caution. This change seems to be the best option, given that people believe we are being intentionally deceptive in keeping them online.

Finally, leadership at ACE all agree it has been too long since we have updated our Mistakes page, so we have added new entries concerning issues we have reflected upon as an organization.

We also notice that there is concern among the community that our recommendations are suspect due to the weak evidence supporting our cost-effectiveness estimates of leafleting. The focus on leafleting for this criticism is confusing to us, as our cost-effectiveness estimates address many interventions, not only leafleting, and the evidence for leafleting is not much weaker than other evidence available about animal advocacy interventions. On top of that, cost-effectiveness estimates are only a factor in one of the seven criteria used in our evaluation process. In most cases, we don’t think that they have changed the outcome of our evaluation decisions. While we haven’t come up with a solution for clarifying this point, we always welcome and are appreciative of constructive feedback.

We are committed to honesty, and are disappointed that the content we've published on the website concerning leafleting has caused so much confusion as to lead anyone to believe we are intentionally deceiving our supporters for profit. On a personal note, I’m devastated to hear that our error in communication has led to the character assassination not only of ACE, but of the people who comprise the organization—some of the hardest working, well-intentioned people I’ve ever worked with.

Finally, I would like everyone to know that we sincerely appreciate the constructive feedback we receive from people within and beyond the EA movement.

*Edited to add links

Issue 2: Running critical pieces by the people you're criticizing is necessary, if you want a good epistemic culture. (That said, waiting indefinitely for them to respond is not required. I think "wait a week" is probably a reasonable norm)

Reasons and considerations:

a) they may have already seen and engaged with a similar form of criticism before. If that's the case, it should be the critic's responsibility to read up on it, and make sure their criticism is saying something new. Or, that it's addressing the latest, best thoughts on the part of the person-being-criticized. (See Eliezer's 4 layers of criticism)

b) you may not understand their reasons well. Especially with something off-the-cuff on facebook. The principle of charity is crucial because our natural tendency is to engage with weaker versions of ideas.

c) you may be wrong about things. Because our kind have trouble cooperating because we tend to criticize a lot, it's important for criticism of Things We Are Currently Trying to Coordinate On to be made as-accurate-as-possible through private channels before unleashing the storm.

Controversial things are intrinsically "public facing" (see: Scott Alexander's post on Trump that he specifically asked people not to share and disabled comments on, but which Ann Coulter ended up retweeting). Because it is controversial it may end up being people's first exposure to Effective Altruism.

Similar to my issue 1, I think Sarah intended this post as tit-for-tat punishment for EA Establishment not responding enough to criticism. Assuming I'm correct about that, I disagree with it on two grounds:

• I frankly think Ben Todd's post (the one most contributing to "EA establishment is defecting on meta-level epistemic discourse norms") was totally fine. GWWC has limited time. Everyone has limited time. Dealing with critics is only one possible use of their time and it's not at all obvious to me it's the best one on the margin. Ben even notes a possible course of action: communicate better about discussions GWWC has already had.

• Even in the maximally uncharitable interpretation of Ben's comments... it's still important to run things by People You Are Criticizing Who Are In the Middle of a Project That Needs Coordination, for the reasons I said. If you're demanding time/attention/effort on the part of people running extensive projects, you should put time/effort into making sure your criticism is actually moving the dialog forward.

I note Constantin's post, first, was extraordinary uncharitable and inflammatory (e.g. the title for the section discussing Wiblin's remark "Keeping promises as a symptom of Autism", among many others); second, these errors were part of a deliberate strategy to 'inflame people against EA'; third, this strategy is hypocritical given the authors (professed) objections to any hint of 'exploitative communication'. Any of these in isolation is regrettable. In concert they are contemptible.

{ETA: Although in a followup post Constantin states her previous comments which were suggestive of bad faith were an "emotional outburst", it did not reflect her actual intentions either at the time of writing or subsequently.}

My view is that, akin to Hostadter's law, virtues of integrity are undervalued even when people try to account for undervaluing them: for this reason I advocate all-but lexical priority to candour, integrity, etc. over immediate benefits. The degree of priority these things should be accorded seems a topic on which reasonable people can disagree: I recommend Elmore's remarks as a persuasive defence of according these virtues a lower weight.

'Lower', however, still means 'quite a lot': if I read Elmore correctly, her view is not one can sacrifice scrupulously honest communication for any non-trivial benefit, but that these norms should on occasion be relaxed if necessary to realise substantial gains. The great majority of EAs seem to view these things as extremely important, and the direction of travel appears to me that 'more respected' EAs tend to accord these even greater importance (see MacAskill; c.f. Tsipursky).

My impression is that EAs - both individually and corporately - do fairly well in practice as well as principle. As Naik notes, many orgs engage in acts of honesty and accountability supererogatory to secure funding. When they do err, they tend to be robustly challenged (often by other EAs), publicly admit their mistake, and change practice (all of the examples Constantin cites were challenged at the time, I also think of Harris's concerns with promotion of EA global, GPP's mistaken use of a Stern report statistic, and now ACE). Similar sentiments apply to an individual level: my 'anecdata' is almost the opposite of Fluttershy's extremely bad experience: I sincerely believe (and even more sincerely hope) that mine is closer to the norm than theirs.

In absolute terms, I don't think EA in toto has a 'lying problem' (or a 'being misleading', 'not being scrupulously honest' problem). It seems to do quite well at this, and the rate and severity of the mistakes I see don't cause great alarm (although it can and should do better). Although relative terms are less relevant, I think it does better than virtually any other group I can think of.

I offer some further remarks on issues raised by some of the examples given which do not fit into the 'lying problem' theme:

1) Ironic, perhaps, that the best evidence for Todd's remark on the 'costs of criticism' arise from the aftermath of a post which (in part) unjustly excoriates him for that particular remark. My impression is that bad criticism is on average much more costly than bad praise, and some asymmetry in how these are treated seem reasonable.

I do not know whether journalistic 'best practice' around 'right of reply' extends to providing the criticism in full to its subject - regardless, it seems good practice to adopt for the reasons Todd explains. I have done this with my co-contributors re. Intentional Insights, and I have run a (yet to be published) piece about MIRI by MIRI as it had some critical elements to it. Naturally, if a critic does not do this for whatever reason, it does not mean their criticism should be ignored (I have yet to see a case of criticism 'shunned' for these reasons) but I think this is a norm worth encouraging.

2) Nonetheless, it may not have been advisable for the head of one 'part' of CEA to bring this up in context of criticism addressed to another part of CEA. Issues around appropriate disclosure have been mentioned before. In addition, remarks by 'EA public figures' may be taken as indicative of the view of their organisations or EA in toto even if explicitly disclaimed as 'personal opinion only'. A regrettable corollary (as Gordon-Brown notes) is a chilling effect on 'EA public figures' refraining from making unguarded remarks publicly. The costs of not doing so may be worse: if EA grows further, we may collectively regret 'providing more ammunition' to external critics to misuse.

3) Given the social costs towards an individual critic, there may be benefit (by organisations or, better, an independent 'Grand Inquisition' collaboration) to canvass these anonymously. The commonest shared could then be explored further: this would be valuable whether they point to a common misconception or a common fault. In the meanwhile, anyone is welcome to disclose criticisms or concerns to me in confidence.

4) Certain practices could be more widely adopted by EA orgs - beyond recording predictions, a prominent 'mistakes' page (per Givewell) would be desirable, likewise scrupulous declaration of relevant conflicts of interests.

5) (I owe this to Carl Shulman). Donors could also pitch in by carefully evaluating empirical or normative claims made by particular EA organisations: Plant, Dickens, and Hoffman would all be laudable examples, and I hope both to contribute some of my own work to this genre and to encourage others to do likewise.

The post does raise some valid concerns, though I don't agree with a lot of the framing. I don't think of it in terms of lying. I do, however, see that the existing incentive structure is significantly at odds with epistemic virtue and truth-seeking. It's remarkable that many EA orgs have held themselves to reasonably high standards despite not having strong incentives to do so.

In brief:

• EA orgs' and communities' growth metrics are centered around numbers of people and quantity of money moved. These don't correlate much with epistemic virtue.
• (more speculative) EA orgs' donors/supporters don't demand much epistemic virtue. The orgs tend to hold themselves to higher standards than their current donors.
• (even more speculative; not much argument offered) Even long-run growth metrics don't correlate too well with epistemic virtue.
• Quantifying (some aspects of) quality and virtue into metrics seems to me to have the best shot at changing the incentive structure here.

The incentive structure of the majority of EA-affiliated orgs has centered around growth metrics related to number of people (new pledge signups, number of donors, number of members), and money moved (both for charity evaluators and for movement-building orgs). These are the headline numbers they highlight in their self-evaluations and reports, and these are the numbers that people giving elevator pitches about the orgs use ("GiveWell moved more than $100 million in 2015" or "GWWC has (some number of hundreds of millions) in pledged money"). Some orgs have slightly different metrics, but still essentially ones that rely on changing the minds of large numbers of people: 80,000 Hours counts Impact-Adjusted Significant Plan Changes, and many animal welfare orgs count numbers of converts to veganism (or recruits to animal rights activism) through leafleting.

These incentives don't directly align with improved epistemic virtue! In many cases, they are close to orthogonal. In some cases, they are correlated but not as much as you might think (or hope!).

I believe the incentive alignment is strongest in cases where you are talking about moving moderate to large sums of money per donor in the present, for a reasonable number of donors (e.g., a few dozen donors giving hundreds of thousands of dollars). Donors who are donating those large sums of money are selected for being less naive (just by virtue of having made that much money) and the scale of donation makes it worth their while to demand high standards. I think this is related to GiveWell having relatively high epistemic standards (though causality is hard to judge).

With that said, the organizations I am aware of in the EA community hold themselves to much higher standards than (as far I can make out) their donor and supporter base seems to demand of them. My guess is that GiveWell could have been a LOT more sloppy with their reviews and still moved pretty similar amounts of money as long as they produced reviews that pattern-matched a well-researched review. (I've personally found their review quality improved very little from 2014 to 2015 and much more from 2015 to 2016; and yet I expect that the money moved jump from 2015 to 2016 will be less, or possibly even negative). I believe (with weaker confidence) that similar stuff is true for Animal Charity Evaluators in both directions (significantly increasing or decreasing review quality won't affect donations that much). And also for Giving What We Can: the amount of pledged money doesn't correlate that well with the quality or state of their in-house research.

The story I want to believe, and that I think others also want to believe, is some version of a just-world story: in the long run epistemic virtue ~ success. Something like "Sure, in the short run, taking epistemic shortcuts and bending the truth leads to more growth, but in the long run it comes back to bite you." I think there's some truth to this story: epistemic virtue and long-run growth metrics probably correlate better than epistemic virtue and short-run growth metrics. But the correlation is still far from perfect.

My best guess is that unless we can get a better handle on epistemic virtue and quantify quality in some meaningful way, the incentive structure problem will remain.

This issue is very important to me, and I stopped identifying as an EA after having too many interactions with dishonest and non-cooperative individuals who claimed to be EAs. I still act in a way that's indistinguishable from how a dedicated EA might act—but it's not a part of my identity anymore.

I've also met plenty of great EAs, and it's a shame that the poor interactions I've had overshadow the many good ones.

Part of what disturbs me about Sarah's post, though, is that I see this sort of (ostensibly but not actually utilitarian) willingness to compromise on honesty and act non-cooperatively more in person than online. I'm sure that others have had better experiences, so if this isn't as prevalent in your experience, I'm glad! It's just that I could have used stronger examples if I had written the post, instead of Sarah.

I'm not comfortable sharing examples that might make people identifiable. I'm too scared of social backlash to even think about whether outing specific people and organizations would even be a utilitarian thing for me to do right now. But being laughed at for being an "Effective Kantian" because you're the only one in your friend group who wasn't willing to do something illegal? That isn't fun. Listening to hardcore EAs approvingly talk about how other EAs have manipulated non-EAs for their own gain, because doing so might conceivably lead them to donate more if they had more resources at their disposal? That isn't inspiring.

My thoughts on this are too long for a comment, but I've written them up here - posting a link in the spirit of making this forum post a comprehensive roundup: http://benjaminrosshoffman.com/honesty-and-perjury/

I have very mixed feelings about Sarah's post; the title seems inaccurate to me, and I'm not sure about how the quotes were interpreted, but it's raised some interesting and useful-seeming discussion. Two brief points:

• I understand what causes people to write comments like "lying seems bad but maybe it's the best thing to do in some cases", but I don't think those comments usually make useful points (they typically seem pedantic at best and edgy at worst), and I hope people aren't actually guided by considerations like those. Most EAs I work with, AFAICT, strive to be honest about their work and believe that this is the best policy even when there are prima facie reasons to be dishonest. Maybe it's worth articulating some kind of "community-utilitarian" norms, probably drawing on rule utilitarianism, to explain why I think honesty is the best policy?
• I think the discussion of what "pledge" means to different people is interesting; a friend pointed out to me that blurring the meaning of "pledge" into something softer than an absolute commitment could hurt my ability to make absolute commitments in the future, and I'm now considering ways to be more articulate about the strength of different commitment-like statements I make. Maybe it's worth picking apart and naming some different concepts, like game-theoretic cooperation commitments, game-theoretic precommitments (e.g. virtues adopted before a series of games is entered), and self-motivating public statements (where nobody else's decisions lose value if I later reverse my statement, but I want to participate in a social support structure for shared values)?

One very object-level thing which could be done to make longform, persistent, not hit-and-run discussion in this particular venue easier: Email notifications of comments to articles you've commented in.

There doesn't seem to be a preference setting for that, and it doesn't seem to be default, so it's only because I remember to come check here repeatedly that I can reply to things. Nothing is going to be as good at reaching me as Facebook/other app notifications on my phone, but email would do something.

Copying my post from the Facebook thread:

Some of the stuff in the original post I disagree on, but the ACE stuff was pretty awful. Animal advocacy in general has had severe problems with falling prey to the temptation to exaggerate or outright lie for a quick win today. especially about health, and it's disturbing that apparently the main evaluator for the animal rights wing of the EA movement has already decided to join it and throw out actually having discourse on effectiveness in favour of plundering their reputation for more donations today. A mistake is a typo, or leaving something up accidentally, or publishing something early by accident, and only mitigation if corrective action was taken once detected. This was at the minimum negligence, but given that it's been there for years without making the trivial effort to fix it should probably be regarded as just a lie. ACE needs replacing with a better and actually honest evaluator.

One of the ways this negatively impacted the effectiveness discourse: During late 2015 there was an article written arguing for ethical offsetting of meat eating (http://slatestarcodex.com/.../vegetarianism-for-meat-eaters/), but it used ACE's figures, and so understated the amounts people needed to donate by possibly multiple orders of magnitude.

More concerning is the extent to which the (EDIT: Facebook) comments on this post and the previously cited ones go ahead and justify even deliberate lying, "Yes, but hypothetically lying might be okay under some circumstances, like to save the world, and I can't absolutely prove it's not justified here, so I'm not going to judge anyone badly for lying", as with Bryd's original post as well. The article sets out a pretty weak case for "EA needs stronger norms against lying" aside for the animal rights wing, but the comments basically confirm it.

I know that answering "How can we build a movement that matches religious movements in output (http://lesswrong.com/.../can_humanism_match_religions.../), how can we grow and build effectiveness, how can we coordinate like the best, how can we overcome that people think that charity is a scam?" with "Have we considered /becoming pathological liars/? I've not proven it can't work, so let's assume it does and debate from there" is fun and edgy, but it's also terrible.

I can think of circumstances where I'd void my GWWC pledge; if they ever pulled any of this "lying to get more donations" stuff, I'd stick with TLYCS and a personal commitment but leave their website.

Good piece

While I think it's good to expect people to have read the same central set of works, I do think we lose out by not being able to snythesisynthesisese discussions. Why isn't there a single community post with the state of the art on this discussion and where key disagreements are? It's understandable that I should have to find the various articles, but why not make it easier for me?

In the interest of completeness: Sarah posted a follow-up on her post. Reply to Criticism on my EA Post.

Since there are so many separate discussions surrounding this blog post, I'll copy my response from the original discussion:

I’m grateful for this post. Honesty seems undervalued in EA.

An act-utilitarian justification for honesty in EA could run along the lines of most answers to the question, “how likely is it that strategic dishonesty by EAs would dissuade Good Ventures-sized individuals from becoming EAs in the future, and how much utility would strategic dishonesty generate directly, in comparison?” It’s easy to be biased towards dishonesty, since it’s easier to think about (and quantify!), say, the utility the movement might get from having more peripheral-to-EA donors, than it is to think about the utility the movement would get from not pushing away would-be EAs who care about honesty.

I’ve [rarely] been confident enough to publicly say anything when I’ve seen EAs and ostensibly-EA-related organizations acting in a way that I suspect is dishonest enough to cause significant net harm. I think that I’d be happy if you linked to this post from LW and the EA forum, since I’d like for it to be more socially acceptable to kindly nudge EAs to be more honest.

I was definitely disappointed to see that post by Sarah. It seemed to defect from good community norms such as attempting to generously interpret people in favour of quoting people out of context. She seems to be applying such rigourous standards to other people, yet applying rather loose standards to herself.

I was overall a bit negative on Sarah's post, because it demanded a bit too much attention, (e.g. the title), and seemed somewhat polemic. It was definitely interesting, and I learned some things.

I find the most evocative bit to be the idea that EA treats outsiders as "marks".
This strikes me as somewhat true, and sadly short-sighted WRT movement building. I do believe in the ideas of EA, and I think they are compelling enough that they can become mainstream.

Overall, though, I think it's just plain wrong to argue for an unexamined idea of honesty as some unquestionable ideal. I think doing so as a consequentialist, without a very strong justification, itself smacks of disingenuousness and seems motivated by the same phony and manipulative attitude towards PR that Sarah's article attacks.

What would be more interesting to me would be a thoughtful survey of potential EA perspectives on honesty, but an honest treatment of the subject does seem to be risky from a PR standpoint. And it's not clear that it would bring enough benefit to justify the cost. We probably will all just end up agreeing with common moral intuitions.

As for the issue of acquiring power/money/influence and then using it to do good it is important to be precise here and distinguish several questions:

1) Would it be a good thing to amass power/wealth/etc.. (perhaps deceptively) and then use those to do good?

2) Is it a good thing to PLAN to amass power/wealth/etc.. with the intention of "using it to do X" where X is a good thing.

2') Is it a good thing to PLAN to amass power/wealth/etc.. with the intention of "using it to do good".

3) Is it a good idea to support (or not object) to others who profess to be amassing wealth/power/etc.. to do good

Once broken down this way it is clear that while 1 is obviously true 2 and 3 aren't. Lacking the ability to perfectly bind one's future self means there is always the risk that you will instead use your influence/power for bad ends. 2' raises further concerns as to whether what you believe to be good ends really are good ends. This risk is compounded in 3 by the possibility that the people are simply lying about the good ends.

Once we are precise in this way it is clear that it isn't the in principle approval of amassing power to do good that is at fault but rather the trustworthiness/accuracy of those who undertake such schemes that is the problem.

Having said this some degree of amassing power/influence as a precursor to doing good is probably required. The risks simply must be weighed against the benefits.

Why Our Kind Can't Cooperate (Eliezer Yudkowsky)

Note to casual viewers that the content of this is not what the title makes it sound like. He's not saying that rationalists are doomed to ultimately lie and cheat each other. Just that here are some reasons why it's been hard.

From the recent Sarah Constantin post

Wouldn’t a pretty plausible course of action be “accumulate as much power and resources as possible, so you can do even more good”?

Taken to an extreme, this would look indistinguishable from the actions of someone who just wants to acquire as much power as possible for its own sake. Actually building Utopia is always something to get around to later; for now you have to build up your strength, so that the future utopia will be even better.

Lying and hurting people in order to gain power can never be bad, because you are always aiming at the greater good down the road, so anything that makes you more powerful should promote the Good, right?

Obviously, this is a terrible failure mode.

I don't buy this logic. Obviously there's a huge difference between taking power and then expending effort into positive activities, or taking power and not giving it up at all. Suppose that tomorrow we all found out that a major corporation was the front for a shady utilitarian network that had accumulated enough power and capital to fill all current EA funding gaps, or something like that. Since at some point you actually do accomplish good, it's clearly not indistinguishable.

I mean, you can keep kicking things back and say "why not secretly acquire MORE power today and wait till tomorrow, and then you'll never do any good?" but there's obvious empirical limitations to that, and besides it's a problem of decision theory which is present across all kinds of things and doesn't have much to do with gaining power in particular.

In practical terms, people (not EAs) who try to gain power with future promises of making things nicer are often either corrupt or corruptible, so we have that to worry about. But it's not sufficient to show that the basic strategy doesn't work.

...

{epistemic status: extremely low confidence}

The way I see a lot of these organizational problems where they seem to have controversial standards and practices is that core people are getting just a little bit too hung up on EA This and EA That and Community This and Community That... in reality what you should do is take pride in your organization, those few people and resources you have in your control or to your left and right, and make it as strong as possible. Not by cheating to get money or anything, but by fundamentally adhering to good principles of leadership, and really taking pride in it (without thinking about overall consequences all the time). If you do that, you probably won't have these kinds of problems, which seem to be kind of common whenever the organization itself is made subservient to some higher ideal (e.g. cryonics organizations, political activism, religions). I haven't been inside these EA organizations so I don't know how they work, but I know how good leadership works in other places and that's what seems to be different. It probably sounds obvious that everyone in an EA organization should run it as well as they can, but after I hear about these occasional issues I get the sense that it's kind of important to just sit and meditate on that basic point instead of always talking about the big blurry community.

To succeed at our goals:

I'd agree with all that. It all seems pretty reasonable.

Note that the proposed norm within EA of following laws at least in the US is very demanding-see this article. A 14th very common violation I would add is not fully reporting income to the government like babysitting: "under the table" or "shadow economy." A 15th would be pirated software/music. Interestingly, lying is not illegal in the US, though lying under oath is. So perhaps what we mean is be as law-abiding as would be socially acceptable to most people? And then for areas that are more directly related to running organizations (not e.g. speeding or jaywalking or urinating outside), we should have a significantly higher standard than the law to preserve our reputation?

It seems to me that a great deal of this supposed 'problem' is simply the unsurprising and totally human response to feeling that an organization you have invested in (monetarily, emotionally or temporally) is under attack and that the good work it does is in danger of being undermined. EVERYONE on facebook engages in crazy justificatory dances when their people are threatened.

It's a nice ideal that we should all nod and say 'yes that's a valid criticism' when our baby is attacked but it's not going to happen. There is nothing we can do about this aspect so let's instead simply focus on avoiding the kind of unjustified claims that generated the trouble.

Of course, it is entirely possible that some level of deception is necessary to run a successful charity. I'm sure a degree of at least moral coercion is, e.g., asking people for money in circumstances it would look bad not to give. However, I'm confident this can be done in the same way traditional companies deceive, i.e. by merely creating positive associations and downplaying negative ones rather than outright lying.

Sarah's post highlights some of the essential tensions at the heart of Effective Altruism.

Do we care about "doing the most good that we can" or "being as transparent and honest as we can"? These are two different value sets. They will sometimes overlap, and in other cases will not.

And please don't say that "we do the most good that we can by being as transparent and honest as we can" or that "being as transparent and honest as we can" is best in the long term. Just don't. You're simply lying to yourself and to everyone else if you say that. If you can't imagine a scenario where "doing the most good that we can" or "being as transparent and honest as we can" are opposed, you've just suffered from a failure mode by flinching away from the truth.

So when push comes to shove, which one do we prioritize? When we have to throw the switch and have the trolley crush either "doing the most good" or "being as transparent and honest as we can," which do we choose?

For a toy example, say you are talking to your billionaire uncle on his deathbed and trying to convince him to leave money to AMF instead of his current favorite charity, the local art museum. You know he would respond better if you exaggerate the impact of AMF. Would you do so, whether lying by omission or in any other way, in order to get much more money for AMF, given that no one else would find out about this situation? What about if you know that other family members are standing in the wings and ready to use all sorts of lies to advocate for their favorite charities?

If you do not lie, that's fine, but don't pretend that you care about doing the most good, please. Just don't. You care about being as transparent and honest as possible over doing the most good.

If you do lie to your uncle, then you do care about doing the most good. However, you should consider at what price point you will not lie - at this point, we're just haggling.

The people quoted in Sarah's post all highlight how doing the most good sometimes involves not being as transparent and honest as we can (including myself). Different people have different price points, that's all. We're all willing to bite the bullet and sometimes send that trolley over transparency and honesty, whether questioning the value of public criticism such as Ben or appealing to emotions such as Rob or using intuition as evidence such as Jacy, for the sake of what we believe is the most good.

As a movement, EA has a big problem with believing that ends never justify the means. Yes, sometimes ends do justify the means - at least if we care about doing the most good. We can debate whether we are mistaken about the ends not justifying the means, but using insufficient means to accomplish the ends is just as bad as using excessive means to get to the ends. If we are truly serious about doing the most good as possible, we should let our end goal be the North Star, and work backward from there, as opposed to hobbling ourselves by preconceived notions of "intellectual rigor" at the cost of doing the most good.