Summary

In 2024, we collected our standard sample for the AI, Morality, and Sentience (AIMS) survey with nationally representative data alongside a secondary sample from Prolific, a popular study recruitment platform. We found substantial differences, suggesting that Prolific data collection cannot readily substitute for nationally representative data in this context. Although the Prolific sample closely aligned with census estimates for age and sex, and approximated on race/ethnicity and region, the sample was more highly educated, more liberal, and reported more experiences with AI than the nationally representative sample—among other general differences. In terms of AI attitudes, two important differences were that Prolific users expressed less moral concern for AI and lower perceived risks from AI even after controlling for demographic effects.

Introduction

Beginning in 2021, the Artificial Intelligence, Morality, and Sentience (AIMS) survey series measures the moral and social perceptions of different types of artificial intelligences (AIs), particularly sentient AIs, in the U.S. public. Relative to our preregistered expectations in 2021, we have observed surprisingly high mind perception and moral consideration. Additionally, the general U.S. public expects sentient AI, human-like AI systems, general intelligence, and superintelligence to emerge within 2–5 years. We have also observed strong support for AI regulation, notably support for banning the development of artificial sentience (62% in 2023, 58% in 2021), and a high degree of perceived risk from advanced AI. 

These observations come from comprehensive nationally representative samples recruited by iSay/Ipsos, Dynata, Disqo, and other leading panels (hereafter, “Ipsos”) to be representative by age, sex, region, race/ethnicity, education, and income based on census estimates from the American Community Survey (ACS). Much digital minds research (e.g., mind perception and morality, AI autonomy and sentience, digital sentience skepticism) uses samples recruited from Prolific, a platform recognized for high-quality, rapid data collection offering general population convenience sampling and limited nationally representative sampling (i.e., quotas for age, sex, race/ethnicity, and political affiliation). 

In 2024, we compared Ipsos and Prolific sample responses on the AIMS survey. To do this, we surveyed 1,099 U.S. American adults from Ipsos and 1,510 U.S. American adults from Prolific (for which we selected the “Representative Sample” option) in November and December 2024. This was the preregistered third wave of AIMS. See the 2023 and 2021 reports for the AIMS methodology. The 2024 analysis code is on the Open Science Framework (OSF). The AIMS data from the Ipsos recruitment can be downloaded from Mendeley Data. The AIMS data from the Prolific recruitment can be downloaded from the OSF.

Comparison of Ipsos and Prolific Samples

We first present the significant differences between Ipsos and Prolific on individual items weighted by the U.S. census, then raw demographic differences, and then raw differences in experiences with AIs. 

Then, we deconstruct the effects of Ipsos and Prolific by demographic differences on the 2021 index variables for moral consideration and social integration (i.e., Mind Perception, AI Moral Concern, AS Treatment, AS Caution, Pro-AS Activism, Perceived Threat).

Note. The tables and figures are optimized for viewing on a larger screen like a laptop or desktop computer rather than a smaller screen like a mobile phone or tablet.

Significant Differences on AIMS Items

Table 1 shows the statistically significant differences between Ipsos and Prolific on individual item responses weighted by the U.S. census alongside aggregate responses (e.g., percentage of agreement, mean, median, or proportion) for the 2021 and 2023 waves and the original research team’s 2021 predictions. 

Table 1: 2021-2024 Weighted Response Comparison

See the interactive table here.

Note. Actual response is 1) % agreement where “agreement” is “Somewhat agree,” “Agree,” “Strongly agree” out of all responses other than “No opinion,” 2) % of people who selected “Yes,” or 3) the mean response. Medians are reported instead of means for two open-ended forecasting items: “If you had to guess, how many years from now do you think that robots/AIs will be sentient?” and “If you had to guess, how many years from now do you think that the welfare of robots/AIs will be an important social issue?” The statistical significance tests are weighted t-tests for continuous and Likert-style variables with means (e.g., PMC1), weighted χ² tests for categorical variables (e.g., F1), and weighted Mann-Whitney U tests for continuous variables with medians (e.g., F2).

The Prolific and Ipsos samples statistically significantly differed on 82% of the AIMS items. Notably, the Prolific sample showed less moral concern for AI and lower perceived risks from AI than the Ipsos sample.

Demographic Differences

Table 2 shows the raw demographic statistics for the Ipsos and Prolific samples alongside the U.S. census estimates.

Table 2: Demographic Characteristics

Census demographics: Whereas Ipsos closely aligned on all census categories (i.e., age, sex, race/ethnicity, region, education, income), Prolific closely aligned with the census data on age and sex, and approximated on race/ethnicity (categories for which Prolific sets quotas if one selects the “Representative Sample” option) as well as region. The Prolific sample was more educated and lower-income than the census data.

Non-census demographics: Ipsos and Prolific differed on three demographic characteristics not tracked by the census. The Prolific sample was more liberal (2.83) than the Ipsos sample (3.14). Of religious affiliations present in greater than 5% of the sample, Prolific (36%) had more non-religious people (i.e., people who report no religion, agnosticism, or atheism) than Ipsos (22%). The Prolific sample also reported a lower percentage of meat-eating diets (86%) than the Ipsos sample (90%).

Different Experiences with AIs

Table 3 shows the raw descriptive statistics for self-reported experiences with AI systems.

Table 3: Experiences with AI 

The Prolific sample was more familiar with AI than the Ipsos sample. There were also contrasts in beliefs about AI sentience. More of the Ipsos sample believed that sentient AIs already exist (17%) than the Prolific sample (11%).

Deconstructing Sample Differences

We next explored differences between Ipsos and Prolific on the moral consideration and social integration index variables (i.e., Mind Perception, AI Moral Concern, AS Treatment, AS Caution, Pro-AS Activism, Perceived Threat), and the extent to which sample differences may be explained by demographic differences. 

To do this, we conducted weighted linear regressions to examine differences between Ipsos and Prolific (Table 4), examined sample effects alongside demographic effects (Table 5), and we estimated how much demographics (e.g., age, sex, exposure to AI) contribute to the observed sample differences (Table 6). 

Table 4: OLS Regressions of Index Variables on Sample

Note. We present the unstandardized beta, standard error and confidence interval associated with the beta, t-statistic, and the uncorrected p-value. Larger betas indicate a stronger effect of the predictor on the outcome, with the sign interpreted like for correlations. Sample: 0 = Ipsos, 1 = Prolific. 

The Prolific sample was significantly lower than the Ipsos sample on the Moral Consideration, Practical Moral Consideration, and Social Integration indices.

Table 5 shows the comparison between the Ipsos and Prolific samples alongside the effects of traditional (e.g., age, sex) and AI-relevant (e.g., owning a robotic device, exposure to AI products and narratives) demographics. By including sample and demographics, we demonstrate that both sample and demographics have unique impacts on the moral and social perceptions of AI.

Table 5: OLS Regressions of Index Variables on Demographics

See the interactive table here.

Note. We present the unstandardized beta, standard error and confidence interval associated with the beta, t-statistic, and the uncorrected p-value. Significance (p) values that became nonsignificant following the FDR correction across predictors in the model are highlighted in grey. Larger betas indicate a stronger effect of the predictor on the outcome, with the sign interpreted like for correlations. Sample: 0 = Ipsos, 1 = Prolific. 

Next, we deconstructed these effects to examine how much demographic differences explain the sample differences on the moral consideration and social integration index variables. Table 6 shows the contributions of select demographics using the Oaxaca-Blinder Decomposition method (Hlavac, 2022; Jann, 2008). 

Larger magnitude percentages indicate stronger contributions to explaining the difference between samples. 

The sign (positive or negative) shows whether differences in that variable work in the same direction as the overall gap (defined as the Ipsos mean minus the Prolific mean). A positive percentage indicates that the demographic differences between the samples on that variable contribute to a higher mean score for Ipsos relative to Prolific, effectively widening the gap. A negative percentage indicates that demographic differences between the samples work against the observed gap, effectively narrowing it (or suggesting that Prolific should actually score higher than Ipsos given that demographic effect).

For example, in Table 6, differences in exposure to AI showed a positive contribution to the sample gap for AS Caution (22.4%). This means that the demographic differences in AI exposure (i.e., Prolific respondents’ average exposure to AI was higher than Ipsos respondents) work in the same direction as the observed gap (i.e., Ipsos’ higher mean than Prolific’s). 

In contrast, differences in exposure to AI show a negative contribution to the sample gap for the other index variables (e.g., Mind Perception = -27.6%, AI Moral Concern = -9.1%, AS Treatment = -42%). For example, this suggests that Prolific could have scored higher than Ipsos on AI Moral Concern because exposure to AI is positively associated with AI Moral Concern, and because exposure to AI is higher in the Prolific sample. Yet, Ipsos still had a higher mean than Prolific. In other words, it appears that other factors outweigh the AI exposure effect, and because much more of the effect is explained by factors external to the model (e.g., Mind Perception = 144.35) than factors in the model (e.g., Mind Perception = -44.35), more of those outweighing factors are likely outside the model. 

Table 6: Demographic Contributions to Sample Differences

Note. Unweighted Blinder-Oaxaca decomposition regressions (Jann, 2008) were employed in R using the “oaxaca” package (Hlavac, 2022). The above demographic contributions explaining the total gap between the samples show the twofold decomposition in which the samples were treated equally. We present contributions of select demographics to index variables for which there was a significant sample difference between Ipsos and Prolific.

In general, we did not find evidence of any AI-related or traditional demographics that reliably explained substantial portions of the gap between Ipsos and Prolific results. Indeed, for most outcomes, the gap between Ipsos and Prolific was due moreso to factors external to our model (e.g., 95.7% of the gap in AI Moral Concern was explained by the factors external to the model). 

One interpretation of this is that it would be challenging to use Prolific data to approximate the nationally representative Ipsos results by adjusting based on demographic differences. It is not as if, for example, we could control for the relatively high AI exposure of Prolific participants or screen only for Prolific participants with lower AI exposure to get an Ipsos-comparable result. Other factors that we did not measure, and complex interactions between factors, may play important roles in explaining the sample gap between Ipsos and Prolific on AI attitudes.

Why Might This Matter?

Effective policymaking and advocacy for AI welfare or safety may change based on whether data is from Prolific (or similar platforms) or nationally representative samples, even when the “Representative Sample” option is selected. For example, we found that, on Prolific, 57% of Americans support a ban on the development of sentient AI, but in the more representative sample, 66% support the ban. Moreover, an advocacy message may appear persuasive in a Prolific sample, but because of the unusually high rates of non-religious people, vegans, vegetarians, and other groups, it might be less persuasive in the general population. 

Appendix

To cite our report please use, https://doi.org/10.31234/osf.io/96y4j_v1, and to cite AIMS data please use, doi.org/10.17632/x5689yhv2n.3.

For our previous AIMS results, please cite one of our three published papers (Anthis et al., 2025; Bullock et al., 2025; Pauketat et al., 2025), our 2023 main or supplemental reports, or our 2021 report.