A recent facebook post by Jeff Kaufman raised concerns about the behavior of Intentional Insights (InIn), an EA-aligned organization headed by Gleb Tsipursky. In discussion arising from this, a number of further concerns were raised.

This post summarizes the concerns found with InIn. It also notes some concerns which were mistaken and unfounded, and facts that arose which reflect well on InIn.

This post was contributed to by Jeff Kaufman, Gregory Lewis, Oliver Habryka, Carl Shulman, and Claire Zabel. They disclose relevant conflicts of interest below.

Outline

1. Exaggerated claims of affiliation or endorsement

Intentional Insights claims 'active collaborations' with a number of Effective Altruist groups in its Theory of Change document which was on its "About" page (August 21, 2016).

In a number of cases InIn makes use of the name of an effective altruist organization without asking for that organization's consent, based on minor interactions such as the organization answering questions about web traffic. From the 'Effective Altruism impact of Intentional Insights' document (August 19, 2016):

As detailed below, we observe that after learning of such claims and use of their names, some of these groups had asked InIn to stop. Yet even in some of these cases InIn had not altered the mentions in its promotional materials months later. Tsipursky also does not appear to have adopted a practice of checking with organizations before using their names in InIn promotional materials.

1.1. Kerry Vaughan of CEA

Tsipursky previously posted notes from a Skype conversation with Kerry Vaughan without his consent, and suggested he had endorsed Intentional Insights where he had not:

Tsipursky later apologized, edited the post, and said he had updated. Yet he later engaged in similar behavior (see sections 1.2 and 1.3 below).

1.2. Giving What We Can (GWWC)

Gleb has taken the Giving What We Can pledge, and contributed an article on the Giving What We Can blog on December 23, 2015. He also mentioned and linked to GWWC in his articles elsewhere.

Michelle Hutchinson, Executive Director of Giving What We Can, wrote to Tsipursky in May 2016 asking him to cease "claiming to be supported by Giving What We Can." However, the use of Giving What We Can's name as an 'active collaboration' was not removed from Intentional Insights' website, and remained in both of the above InIn documents as of October 15, 2016.

1.3. Animal Charity Evaluators (ACE)

In the InIn impact document Tsipursky quotes Leah Edgerton of ACE:

Erika Alonso of ACE subsequently made the following statement:

2. Astroturfing

Astroturfing is giving the misleading impression of unaffiliated ("grassroots") support. In GiveWell's first year its cofounders engaged in astroturfing, and this was taken very seriously by its board. Among other responses, the GiveWell board demoted one of the co-founders and fined both $5,000 each. Tsipursky expressly claimed not to engage in astroturfing:

However, astroturfing is widespread across the Intentional Insights social media presence (documented in the sections below). Tsipursky did qualify his statement with "we are not asking people to do these sorts of activities in their paid time", but lack of payment isn't enough to prevent misleading people about the nature of the support. In any case, the distinction between contractors' paid and unpaid time is blurry (see section 4.1.1).

2.1. The Intentional Insights blog

Paid contractors for Intentional Insights leave complimentary remarks on the Intentional Insights blog, and the Intentional Insights account replies with gratitude, as if the comments were by strangers. At no stage do they disclose the financial relationship that exists between them. In the screenshot below (source, Candice, John, Beatrice, Jojo, and Shyam are all Intentional Insights contractors.

The most recent examples of this happened in late August 2016, after the initial post and discussion with Tsipursky on Jeff's Facebook wall, and during the drafting of this document.

2.2. The Effective Altruism forum

Tsipursky has done the same thing on the Effective Altruism forum. Here is one instance (note that "Nyor" also goes by "Jojo"):

Here is another example (note that "Anthonyemuobo" is a professional handle used by one of Tsipursky's acknowledged contractors, "Sargin"):

2.3. LessWrong

Tsipursky posted a link to some of his wife and InIn co-founder's writing in February 2016, without noting this connection:

This is a minor lapse, one which Gleb claimed to have learned from and updated. Yet similar behavior continued:

In March 2016, Intentional Insights' contractors created accounts and started posting non-specific praise on Tsipursky's LessWrong posts:

These are all people Tsipursky pays, but none of them acknowledged it in their comments or their posts in the welcome thread. Additionally, Tsipursky did not acknowledge this relationship when he thanked them for their remarks.

LessWrong user gjm pointed out that this was misleading, and Tsipursky acknowledged this was a problem and commented on Sargin's welcome post:

While Tsipursky knew both Beatrice Sargin and Alex Wenceslao had posted similar comments, since he had replied to them, he waited for these to be discovered and pointed out before acting:

This happened a third time, with JohnC2015:

2.4. Facebook

2.4.1. Soliciting upvotes and denying it

Tsipursky claimed "when I make a post on the EA Forum and LW I will let people who are involved with InIn know about it, for their consideration, and explicitly don't ask them to upvote":

In the comment Tsipursky denies soliciting upvotes, and demands that accusations that he did be substantiated or withdrawn. Six hours later someone responded with a screenshot of a post Tsipursky had made to the Intentional Insights Insiders group showing Tsipursky soliciting upvotes:

Tsipursky's response, a couple hours later in the same thread:

Tsipursky either genuinely believed posts like the above do not ask for upvotes, or he believed statements that are misleading on common-sense interpretation are acceptable providing they are arguably 'true' on some tendentious reading. Neither is reassuring. [He subsequently conceded this was 'less than fully forthcoming'.]

2.4.2. Not disclosing paid support

Intentional Insights proposed producing EA T-Shirts, and received multiple criticisms. Tsipursky claimed he had run the design by multiple people. Again, Tsipursky did not disclose that at least five of them were people he pays:

2.5. Amazon

Tsipursky's contractor posted a 5-star review for his self-help book on Amazon without disclosing the affiliation:

Tsipursky emailed copies of his self-help book to Intentional Insights volunteers, including contractors, who responded by posting 5-star reviews on Amazon:

He later followed up with:

This is true but incomplete: the 8th review is by Asraful Islam, a volunteer affiliated to Intentional Insights.

Another Intentional Insights affiliate, unpaid at that time but now a paid virtual assistant, Elle Acquino, posted another 5-star review, not in the top 10. In that review, however, the connection to Tsipursky and his nonprofit institute was disclosed.

3. Misleading figures

In December 2015 and January 2016, Tsipursky repeatedly claimed that his articles were shared thousands of times as evidence of the effectiveness of his approach. In fact, he had been reporting Facebook 'likes' and all views on Stumbleupon as shares, greatly exaggerating the extent of social media engagement.

The initial point reflected a common issue with the interpretation of social media activity counters on websites. After this was explained to him Tsipursky claimed to have updated on the correction. However, a June 2016 document on Intentional Insight's Effective Altruism Impact again reported views as shares, exaggerating sharing by many times.

4. Dubious practices

4.1.1. Paid contractors' expected 'volunteering'

Tsipursky only takes on contractors who spend at least two hours "volunteering" for Intentional Insights for each paid hour:

In a follow-up discussion, Tsipursky suggested that contractors could temporarily reduce their volunteer hours in special circumstances, but he would not affirm that contractors would be allowed to simply say no to "volunteering":

Depending on the nature of the volunteer work, this requirement seems potentially unethical, effectively requiring that contractors do three times as much work for a fixed amount of money. We also suggest this relationship undermines the distinction Tsipursky offers between 'paid' and 'volunteer time' and the defence that the promotion his contractors undertake on his behalf is innocuous as it occurs in their 'volunteer time'.

4.1.2. Further details regarding contractor 'volunteering'

Subsequent to the preparation of the above section Tsipursky provided additional information about how he came into contact with contractors, their donations, prior unpaid volunteering, wages, and other information as evidence of genuine support. They provide that, but also support concerns regarding linkage of paid and unpaid work and financial interests.

Tsipursky states the following regarding initial meetings and hiring:

Tsipursky stated the following regarding the length of unpaid volunteering prior to the first paid work:

He also notes donations by contractors, implemented by reducing their paid hours or paid hour wage rate, as evidence of genuine support:

I have pointed out many times that there is plenty of evidence showing that those folks who do contracting are passionate enthusiasts for InIn. Let's take the example of John Chavez, who the document brought up. He chose to respond to a fundraising email to our supporter listserve in June 2016 – long before Jeff Kaufman's original post – by donating $50 per month to InIn out of his $300 monthly salary:

This is bigger than a typical GWWC member, at over 15% of his annual income. Let me repeat – he voluntarily, out of his own volition in response to a fundraising that went out to all of our supporters, chose to make this donation. Just to be clear, we send out fundraising letters regularly, so it’s not like this was some special occasion. It was just that – as he said in the letter – it happened to fall on the 1-year anniversary of him joining InIn and he felt inspired and moved by the mission and work of the organization to give.

Before you go saying John is unique, here is another screenshot of a donation from another contractor who in October 2015, in response to a fundraising email, made a $10/month donation:

Again, voluntarily, out of her own volition, she chose to make this donation.

Tsipursky also indicates that paid and unpaid hours by contractors constitute only a minority of work hours at InIn, with most hours contributed by volunteers without financial compensation:

Regarding wages and requirement/expectations of unpaid volunteering, Tsipursky wrote the following:

The Upwork (formerly known as Odesk) freelancer marketplace on which contractors are hired has a minimum wage of $3.00 per hour. Combined with the expected unpaid volunteering the typical wage would be $1.00, 1/3rd of the minimum for the platform.

John is given as an example of a higher paid contractor at $7.5 per hour. However, this is combined with 3 hours of unpaid volunteering for each paid hour, rather than 2, for a combined wage of $1.875 per hour, prior to his donation of 1/3 of that wage.

In effect, the expectation of volunteering systematically circumvents the Upwork minimum wage for contractors. However, it should be noted that the Upwork minimum wage is a corporate policy, and not a national or local labor law. Contractors in low-income countries may be earning substantially more than the local minimum wages or average incomes. For example, according to Wikipedia the hourly minimum wage in US dollars at nominal exchange rates is $0.54 in Nigeria. In the Philippines minimum wages vary by location and sector, but Wikipedia lists a range of ~$0.6-$1.2 per hour for non-agricultural workers, with the latter group in the capital of Manila. So the wage per combined (paid+volunteer) hour of work would not appear to be in conflict with legal minimum wages in contractors' jurisdictions. Furthermore in a number of these jurisdictions the minimum wage is closer to the median wage, and unemployment is high.

Regarding the link between paid and unpaid hours, Tsipursky describes it as an informal understanding:

In aggregate the additional statements provide evidence of pre-existing support for InIn from new contractors. However, they also confirm a linkage of paid and unpaid labor, and contractor financial interests in promotional activity occurring during 'volunteer' hours.

4.2. "Best-selling author"

Tsipursky includes being a 'best-selling author' in his standard bio. For example, on his Patreon:

And:

And on his Amazon author page:

Normally, a reader would take "best-selling author" to mean hitting a major best-seller list like the New York Times, which indicates that very many people have decided to buy the book, and is a hard signal to fake. In Tsipursky's case, "best-selling author" means that his book was very briefly the top seller in a sub-sub-category of Amazon. Further, he reports offering his book for free and encouraging friends and contractors to download and review it. In its first two weeks the book sold 50 copies at $3 each. Cumulatively it has sold 500 copies at $3 each, and been downloaded 3500+ times free. In contrast, NYT bestseller status requires thousands of sales over the first week. Amazon bestseller status is calculated hourly by category: in small categories three purchases in an hour can win the #1 bestselling author label.

Many of those giving the book 5 star reviews are social contacts of Tsipursky, some of them paid or volunteer Intentional Insights staff, but do not disclose this association (see section 2.5).

As of August 22, 2016 the book is ranked as follows:

In light of this, calling oneself a 'bestselling author' on this sort of performance is potentially misleading.

We note that the practice of claiming bestselling author status using bestseller lists that involve very small actual sales may be widespread. This does not, however, prevent it from being misleading or controversial. For example, when Brent Underwood attained Amazon best-seller status using a few dollars in less than an hour with a book that was simply a picture of his foot, media coverage generally suggested that this highlighted a problematic practice.

5. Inflated social media impact

5.1. Facebook

Tsipursky has cited social media engagement as evidence of impact. However, in many cases it appears that this engagement is illusory. In the case of Facebook, it appears to have resulted from paid Facebook post boosting, which led to hundreds of likes on posts from clickfarms, in a process described by Veritasium: clickfarm accounts like enormous numbers of things they have not been directly paid to like in order to manipulate Facebook’s algorithms. Facebook boosting systematically attracts these clickfarm accounts, a risk which is exacerbated by boosting to regions where clickfarms are located (although clickfarms also have fake accounts purporting to be from all around the world).

In the case of InIn posts, InIn paid for that boosting. In February 2016, Tsipursky argued that this was resulting in genuine engagement and reach:

For a number of InIn blog posts with large numbers of likes (for example 318 for this recent one) these likes appear to be primarily the result of clickfarms. Accounts liking this post like vast numbers of disparate things. Here are some random selections from the middle of the list of that post:

• Himanshu's likes
• Anik's likes
• Debdeep's likes
• Sardar's likes
• Jidan's likes
• Kanhaiya's likes
• Julak's likes
• Farhana's likes
• Kawser's likes
• Selly's likes

There is further circumstantial evidence: the likes are often from accounts in low-income countries with substantial clickfarm operations. Tsipursky defended this as coincidental overlap caused by Intentional Insight's targeting of low-income countries, however countries with similar demographics without large click farm operations are not well represented.

In arguing for the impact of his writing, Tsipursky cited a post on the TLYCS blog that got 500 likes in its first day on the TLYCS blog while typical posts got 100-200 likes:

However, this appears to also be a case of Facebook ad boosting eliciting engagement from clickfarms, this time by a former TLYCS employee (subsequently asked to stop by TLYCS) rather than InIn, according to this statement from TLYCS' Jon Behar:

The profiles contributing the likes and whose profiles show no other engagement with TLYCS, or with EA ideas:

After Jeff Kaufman raised concerns about the pattern of Facebook likes in February 2016, Tsipursky doesn't seem to have looked into the issue further prior to the August 2016 discussion, when outside observers provided indisputable evidence and explained the role of boosting in generating clickfarm likes. While the boosting-clickfarm link is counterintuitive, the lack of any other engagement by the clickfarmers was apparent both before and after the raising of concerns in February. Failure to examine the ineffectiveness of these social media channels, even after concerns were raised, raises questions about InIn's practices as an outreach and content marketing organization.

5.2. The Life You Can Save donations

In his "Effective Altruism impact of Intentional Insights" document (archived copy), Tsipursky claims that content he has published with The Life You Can Save is able to "regularly reach an audience of over 5,000, at least 12% of whom make a donation" suggesting over 600 donations per article, based on a reference letter from a former TLYCS employee. However, these figures were incorrect, and TLYCS estimates that the total number of visitors who landed on Tsipursky's blog posts at the TLYCS blog was ~3,000 (rather than tens of thousands), with donations directly from those page totalling likely 2-3 (rather than hundreds).

While the reference letter Tsipurksy cites could easily give that false impression, it is implausible in light of other information available to him about the impact of his pieces. For example, Tsipursky also cites an article in a major news outlet as producing two donations to GiveDirectly totalling $500:

Since two donations is far less than ~600, this "12% of 5,000 views" number was clearly not sanity checked before being used to argue the case for Intentional Insights to EAs and in a fundraising document aimed at EAs. It's possible that Tsipursky simply took a surprisingly good estimate from a partner organization at face value, but one might expect an expert in marketing to investigate why this channel was performing so much better than his other channels.

5.3. Twitter

Tsipursky implied that his 10k Twitter followers represent organic interest:

The InIn account is following approximately as many accounts as follow it, 11.7k to 11.4k. Oliver observed that many of these accounts have "100% follow-back" in their descriptions. It seems like they're offering an exchange: InIn adds these accounts as followers, and they add his back in return, or vice versa. This is not an indication of actual interest from fans, and these accounts have almost no organic engagement with InIn such as retweets:

5.4. Pinterest

InIn follows over 20,000 people on Pinterest, far more people than follow it. As on the InIn Facebook page and Twitter, follower engagement is extremely low, and dominated by persons affiliated with InIn, suggesting the vast majority of followers are not genuine.

Examining the profiles of followers, there appears to be a very high rate of clickfarm/advertising accounts. Here are 10 randomly selected InIn Pinterest follower accounts. 10 out of 10 appear to be spam/advertising/clickfarm accounts:

• borisparr
• gabrielledowdq
• oradarcusiwz
• beautytips70413
• evareese302
• jacobwelch336
• brandon_clarkso
• healthabundantl
• emmahughes1984
• cassondrapinkha

5.5. Presentations of media article traffic and reach

5.5.1. TIME article

In the InIn EA impact document we see this:

The document does not make clear that the article did not appear in the print magazine, so print readers would not be exposed to it there. Online, we are left to anchor on a figure of 65 million views, without any reference to the actual views of the article (which were tremendously lower).

Somewhat later in the document we see this:

As another example, here are numbers in a spreadsheet we set up recently to track clicks to EA nonprofit websites from the Time piece we published.

However, while the article made the case for GiveWell recommended charities and EA charity evaluators, only 132 clicks reached those organizations through the article, 70 of which did not immediately bounce, according to InIn's traffic figures. Specifically, in the original InIn spreadsheet the 'signed up to newsletter or converted in other ways'' column had a value of 13 for ACE, and 1 'clicked on donate button'

The corrected spreadsheet shows a value of 2 rather than 13 for 'signed up to newsletter.'

Thus InIn knew that the product of traffic and click-through was very low, suggesting some combination of low traffic for a piece on Time's website and low click-through rates. However this negative information was removed from the main text of the document while the 65 million figure (for all articles on the TIME website, including dubious traffic) was made prominent.

5.5.2. Huffington Post

The InIn EA impact document also included this discussion of a Huffington Post article:

However, he provided no evidence of reaching new audiences via the placement in the HuffIngton Post. Instead, he provided an example of an already supportive facebook friend, who apparently encountered the article from Tsipursky's Facebook page, not the Huffington Post.

6. Mistaken/Unfair accusations

6.1. Supposed linearity of Twitter follower increase

It was suggested that Tsipursky's twitter page shows surprisingly linear increases in followers over time (e.g. +8 followers a day for 10 days in a row, which may be indicative of click-farming. This piece of evidence is likely mistaken, as the tool used (sharecounter) probably linearly interpolates days where they do not record a user's Twitter followers, and thus the apparent linearity is an artifact.

6.2. Objections to Intensional Insights staff 'liking' Intentional Insights content

In the course of the original discussion of Jeff's post on Facebook, numerous people took exception to staff or volunteers 'liking' or supporting InIn content. This criticism is misguided: this is common practice both for nonprofits generally and within the EA community: many EAs affiliated to a given group 'like' or share content without disclosing their affiliation. Although issues around appropriate disclosure can be subtle, acts like this on social media do not on reflection seem significant enough to warrant disclosure of interests to the authors of this document.

6.3. 'Paid likes' from clickfarms

In the February 2016 discussion it was suggested the Tsipursky might be directly paying for likes from clickfarms. However, as discussed in section 5.1, while the likes in question appear to have resulted from paid Facebook boosting, and to be from clickfarms, they were not directly paid for. Instead, the boosting attracted clickfarm likes through an accidental process explained well the linked Veritasium video.

7. Positives

In the course of research into and discussion around InIn, some facts that reflect well on InIn were discovered. These are listed below. We don't think this comprises all evidence favourable of InIn: the impact document, Tsipursky's post on the EA forum, and the Intentional Insights website offer further evidence. (We have not looked at these closely enough to have a view on them.)

7.1. Jon Behar

One TLYCS employee who was worked with Tsipursky on Giving Games says Tsipursky has made helpful introductions:

Behar is also quoted in the InIn EA impact doc as saying:

7.2. Additional donations

TLYCS has information indicating that Tsipursky's posts combined drove about two or three donations, and the Huffington Post article resulted in to donations to GiveDirectly totaling $500. Tracking donations is hard, so this is definitely an underestimate.

7.3. Placement of articles in TIME and the Huffington Post

Tsipursky's articles in TIME and the Huffington Post got lots of exposure for EA ideas. Additionally, being able to get articles placed there is impressive.

8. Policy responses from InIn

During discussions with Tsipursky regarding drafts of this document he mentioned some InIn policy changes made in response to the criticisms. This section does not reflect any other changes InIn may have made, primarily because we haven't been able to put in the time to follow up on each practice and see whether it has continued. We also note that Tsipursky provided additional information regarding Amazon sales, contractor names, and payment practices upon request for this document.

8.1. Post-criticism conflict-of-interest policy

Following the discussion under Jeff Kaufman's post in August 2016, InIn created a conflicts of interest policy document:

8.2. Post-criticism Facebook boosting

Tsipursky now states:

Regarding InIn social media policy, we are making sure to avoid boosting any more posts to clickfarm countries. We're generally not boosting posts right now to anyone but fans of the page who live in the US and other rich countries. We found we couldn't ban identifiable clickfarm accounts from the FB page, unfortunately, so we're being really cautious about boosting posts.

9. Disclosures

Many people contributed to this document, some of them anonymously. Below are disclosures from people who contributed substantially and want to be clear about any potential conflicts of interest. None of the individuals below contributed on behalf of an employer or organization, and their contributions should not be taken to imply any stance on the part of any organization with which they are affiliated.

• Jeff Kaufman has donated to the Centre for Effective Altruism (CEA), 80,000 Hours, and Giving What We Can. He has volunteered for Animal Charity Evaluators in a very minor capacity. His wife, Julia Wise, works for CEA and serves on the board of GiveWell.

• Gregory Lewis has previously worked as a volunteer for Giving What We Can and 80,000 hours. He has donated to Giving What We Can and the Global Priorities Project.

• Oliver Habryka currently works for CEA, and has been active in EA community organizing in a variety of roles.

• Carl Shulman currently works for the Future of Humanity Institute, and consults for the Open Philanthropy Project. He previously worked for the Machine Intelligence Research Institute (MIRI). He has previously done some consulting and volunteering for the Center for Effective Altruism, especially 80,000 Hours. His wife is executive director of the Center for Applied Rationality and a board member of MIRI.

• Claire Zabel works at the Open Philanthropy Project, and serves on the board of Animal Charity Evaluators. She has donated to a variety of EA organizations and has close ties with other people in the EA community.

10. Response comments from Gleb Tsipursky

Tsipursky has responded in the comments below: part one, part two, part three.