Comment Permalink

I've just spent about an hour on the survey, at which point I noticed the progress bar was at about 1/6th. This was at the start of four timed questions which required 2 minutes each, with each one having a few follow-up questions.

At this point there had been 7 or 8 of these timed questions, as well as at least two dozen pages of multiple-choice questions and a few 'brain-teasers'. I do not see how it is possible to complete this first 1/6th in under 45 minutes, seeing that the timed questions alone already take up 16 minutes.

I sincerely apologize for the length of the survey.

See in context

Recruiting for Survey on the Psychology of EA

by Kyle Fiore Law

Jan 25 20241 min read 20

25

CommunityOpportunities to take actionBuilding effective altruismResearchSurveys

Frontpage

I am actively recruiting effective altruists to participate in an online survey mapping their psychological profiles. The survey should take no more than 90 minutes to complete and anyone who identifies as being in alignment with EA can participate. If you have the time, my team and I would greatly appreciate your participation! The survey pays $15 and the link can be found below.

Survey Link: https://albany.az1.qualtrics.com/jfe/form/SV_8v31IDPQNq4sKBU

The Research Team:

Kyle Fiore Law (Project Leader; PhD Candidate in Social Psychology; University at Albany, SUNY): https://www.kyleflaw.com/
Brendan O’Connor (Associate Professor of Psychology; University at Albany, SUNY):
Abigail Marsh (Professor of Psychology and Interdisciplinary Neuroscience; Georgetown University)
Liane Young (Professor of Psychology and Neuroscience; Boston College)
Stylianos Syropoulos (Postdoctoral Researcher; Boston College)
Paige Amormino (Graduate Student; Georgetown University)
Gordon Kraft-Todd (Postdoctoral Researcher; Boston College)

Warmly,

Kyle :)

25 Reactions

Comments18

Sorted by

New & upvoted

Click to highlight new comments since: Today at 10:41 AM

Vasco Grilo🔸Jan 27 202423

Thanks for sharing, Kyle!

May I ask how you estimated the time to complete the survey? I gave up because I made very little progress in 40 min, but I might be unusually slow.

I think 15 $ is a minor incentive for 90 min, but I tried to complete the survey anyway because this type of work seems useful. However, my guess is that you would get way more responses if you did a smaller e.g. 15 min survey even it had no monetary incentive. I also wonder whether the responses would be more accurate, because 90 min is so much time that people may start to give inaccurate responses. Alternatively, assuming I am not unusually slow, people may give rushed responses in order to finish the survey in 90 min.

In general, I also felt like many questions were overly similar. I appreciate some of this may be needed for internal validity purposes, but I would say the trade-off as the survey stands is not great.

Sorry if my comments come across as harsh. Thanks for working towards contributing to a better world!

Geoffrey MillerJan 26 202410

I just reposted your X/Twitter recruitment message, FWIW:

https://twitter.com/law_fiore/status/1706806416931987758

Good luck! I might suggest doing a shorter follow-up survey in due course -- 90 minutes is a big time commitment for $15 payment!

gyang74Feb 15 20244

Please I participated in the survey and $15 was sent to my email I tried to send the fund to my PayPal account and the transaction was cancelled. How can I claim my $15

Caleb ZegeMar 7 20241

Hi Gyang, how long did it take for you to get the pay?

Anonymous000Feb 4 20243

Kyle Fiore LawFeb 13 20241

I sincerely apologize for the length of the survey.

Kyle Fiore LawJan 31 20243

Dear All,

I’m truly grateful for the valuable feedback and support received from the community. Below, I’ve provided some clarifications that I hope you’ll find useful.

To help ensure an accurate assessment of survey length, we timed the survey on the basis of median completion time derived from a pilot study with a general population sample (N=320). We’re aware of the concerns about respondent fatigue, which is why the pilot was crucial. It allowed us to examine the response quality and ensure the consistency and validity of our measures, aligning with well-established findings in relevant literature.

This survey marks our initial foray into this research area. As such, we opted for a comprehensive approach in selecting our measures. This approach enables us to explore numerous key variable relationships thoroughly, setting the stage for a more streamlined follow-up study.

Your participation, time and effort in helping us with this project are immensely appreciated. We are hopeful that the insights gained will significantly contribute to the development of more effective EA outreach strategies within the general population.

Warmly,

Kyle :)

Geoffrey MillerJan 31 202410

Kyle - I just completed the survey yesterday. I did find it very long and grueling. I worry that you might get lower quality data in the last 1/2 of the survey, due to participant fatigue and frustration.

My suggestion -- speaking as a psych professor who's run many surveys over the last three decades -- is to develop a shorter survey (no more than 25 minutes) that focuses on your key empirical questions, and try to get a good large sample for that.

Kyle Fiore LawFeb 1 20241

Thank you, Geoffrey! I really appreciate your time and candid feedback. I will take this into careful consideration going forward.

ArepoFeb 3 20242

Others have mentioned the length of the survey, but I think it would also be useful for long surveys to use a survey provider that has a progress bar.

ETA: I now realise there is a progress bar, but I didn't register it because it was advancing so slowly. I rescind my initial implication that others have said adequately expressed the length of this survey...

ArepoFeb 3 20243

Also a 101-point Likert scale seems asking us for overmuch precision :P

ArepoFeb 3 20243

Also I'm on a 7-point Likert scale page where I can only click on the first five options (about how much thought I like my tasks to involve).

I resolved it by shrinking the browser window so it switched to dropdown menus, but some people might not think to do that.

Kyle Fiore LawFeb 13 20243

This is very good to know. Thank you for sharing these insights!

Caleb YasheFeb 20 20241

Hello, i completed the survey 3 days agi and was asked to drop my email to facilitate payment. I haven't heard a feed back from you since

Caleb ZegeMar 7 20241

Hey Caleb,did you later receive the payment and how long did it take?

Caleb YasheApr 18 20242

Hello Caleb,

Yes i did, i reached out to Kyle and it was sorted within 24 hours.

Caleb ZegeMar 7 20241

I completed the survey hours ago, still yet to see any compensation though

Motiloluwa Abiola Oriowo Feb 23 20241

Hello, I participated in the survey and my study participation fund sent to PayPal was refunded back to Brenda O'Connor.

Kindly make my payment. Thanks