DM

David_Moss

Principal Research Director @ Rethink Priorities
8791 karmaJoined Working (6-15 years)

Bio

I am the Principal Research Director at Rethink Priorities. I lead our Surveys and Data Analysis department and our Worldview Investigation Team. 

The Worldview Investigation Team previously completed the Moral Weight Project and CURVE Sequence / Cross-Cause Model. We're currently working on tools to help EAs decide how they should allocate resources within portfolios of different causes, and to how to use a moral parliament approach to allocate resources given metanormative uncertainty.

The Surveys and Data Analysis Team primarily works on private commissions for core EA movement and longtermist orgs, where we provide:

  • Private polling to assess public attitudes
  • Message testing / framing experiments, testing online ads
  • Expert surveys
  • Private data analyses and survey / analysis consultation
  • Impact assessments of orgs/programs

Formerly, I also managed our Wild Animal Welfare department and I've previously worked for Charity Science, and been a trustee at Charity Entrepreneurship and EA London.

My academic interests are in moral psychology and methodology at the intersection of psychology and philosophy.

How I can help others

Survey methodology and data analysis.

Sequences
3

RP US Public AI Attitudes Surveys
EA Survey 2022
EA Survey 2020

Comments
588

Thanks for writing on this important topic!

I think it's interesting to assess how popular or unpopular these views are within the EA community. This year and last year, we asked people in the EA Survey about the extent to which they agreed or disagreed that:

Most expected value in the future comes from digital minds' experiences, or the experiences of other nonbiological entities.

This year about 47% (strongly or somewhat) disagreed, while 22.2% agreed (roughly a 2:1 ratio).

However, among people who rated AI risks a top priority, respondents leaned towards agreement, with 29.6% disagreeing and 36.6% agreeing (a 0.8:1 ratio).[1]

Similarly, among the most highly engaged EAs, attitudes were roughly evenly split between 33.6% disagreement and 32.7% agreement (1.02:1), with much lower agreement among everyone else.

This suggests to me that the collective opinion of EAs, among those who strongly prioritise AI risks and the most highly engaged is not so hostile to digital minds. Of course, for practical purposes, what matters most might be the attitudes of a small number of decisionmakers, but I think the attitudes of the engaged EAs matters for epistemic reasons. 

 

  1. ^

    Interestingly, among people who merely rated AI risks a near-top priority, attitudes towards digital minds were similar to the sample as a whole. Lower prioritisation of AI risks were associated with yet lower agreement with the digital minds item.

Thanks!

Yes, we plan to repeat Pulse at least another three times in the US.

We'd be interested to run it in other countries if there were interest in funding it.

Relatedly, I think in many cases burnout is better conceptualised as depression (perhaps with a specific work-related etiology). 

Whether burnout is distinct from depression at all is a controversy within the literature:

I think that this has the practical implications that people suffering from burnout should at least consider whether they are depressed and consider treatment options with that in mind (e.g. antidepressants, therapy). 

There's a risk that the "burnout" framing limits the options people are considering (e.g. that they need rest / changes to their workplace). At the same time, there's a risk that people underestimate the extent to which environmental changes are relevant to their depression, so changing their work environment should also be considered if a person does conclude they might be depressed.

Answer by David_Moss54
16
0
3

I would like someone to write a post about almost every topic asked about in the Meta Coordination Forum Survey, e.g.

  • What should the growth rate of EA be?
  • How quickly should we spend EA resources?
  • How valuable is recruiting a highly engaged EA to the community?
  • How much do we value highly engaged EAs relative to a larger number of less engaged people hearing about EA?
  • How should we (decide how to) allocate resources across cause areas?
  • How valuable is a junior/senior staff hire at an EA org (relative to the counterfactual second best hire)?
  • What skills / audiences should we prioritise targeting?

I'm primarily thinking about core EA decision-makers writing up their reasoning, but I think it would be valuable for general community members to do this.

Prima facie, it's surprising that more isn't written publicly about core EA strategic questions.

Answer by David_Moss19
0
0
1

Some things you might want to do if you are making a weighted factor model

Weighted factor models are commonly used within EA (e.g. by Charity Entrepreneurship/AIM and 80,000 Hours). Even the formalised Scale, Solvability, Neglectedness framework can, itself, be considered a form of weighted factor model.

However, despite their wide use, weighted factor models often neglect to use important methodological techniques which could test and improve their robustness,  which may threaten their validity and usefulness. RP's Surveys and Data Analysis  team previously consulted for a project who were using a WFM, and helped them understand certain things that were confusing them about the behaviour of their model using these techniques, but we've never had time to write up a detailed post about these methods. Such a post would discuss such topics as:

  • Problems with ordinal measures
  • When (not) to rank scores
  • When and how (not) to normalise
  • How to make interpretable rating scales
  • Identifying the factors that drive your outcomes
  • Quantifying and interpreting disagreement / uncertainty

How to interpret the EA Survey and Open Phil EA/LT Survey.

I think these surveys are complementary and each have different strengths and weaknesses relevant for different purposes.[1] However, I think what the strengths and weaknesses are and how to interpret the surveys in light of them is not immediately obvious. And I know that in at least some cases, decision-makers have had straightforwardly mistaken factual beliefs about the surveys which has mislead them about how to interpret them. This is a problem if people mistakenly rely on the results of only one of the surveys, or assign the wrong weights to each survey, when answering different questions.

A post about this would outline the key strengths and weaknesses of the different surveys for different purposes, touching on questions such as:

  • How much our confidence should change when we have a small sample size from a small population.
  • How concerned we should be about biases in the samples for each survey and what population we should be targeting.
  • How much the different questions in each survey allows us to check and verify the answers within each survey.
  • How much the results of each survey can be verified and cross-referenced with each other (e.g. by identifying specific highly engaged LTists within the EAS).

 

  1. ^

    Reassuringly, they also seem to generate very similar results, when we directly compare them, adjusting for differences in composition, i.e. only looking at highly engaged longtermists within the EA Survey.

Yeh, I definitely agree that asking multiple questions per object of interest to assess reliability would be good. But also agree that this would lengthen a survey that people already thought was too long (which would likely reduce response quality in itself). So I think this would only be possible if people wanted us to prioritise gathering more data about a smaller number of questions.

Fwiw, for the value of hires questions, we have at least seen these questions posed in multiple different ways over the years (e.g. here) and continually produce very high valuations. My guess is that, if those high valuations are misleading, this is driven more by factors like social desirability than difficulty/conceptual confusion. There are some other questions which have been asked in different ways across years (we made a few changes to the wording this year to improve clarity, but aimed to keep the same where possible), but I've not formally assessed how those results differ. 

Thanks Vasco!

This bullet plus the other I quoted above suggest typical junior and senior hires have lifetimes of 40.2 (= 2.04*10^6/(50.7*10^3)) and 16.1 roles (= 7.31*10^6/(455*10^3)), which are unreasonably long. For 3 working-years per junior hire, and 10 working-years per senior hire, they would correspond to working at junior level for 121 years (= 40.2*3), and at senior level for 161 years (= 16.1*10).

We took a different approach to this here, where we looked at the ratio between the value people assigned to a role being filled at all and the value of a person joining the community, rather than the value of the first vs second most preferred hire.

If we look at those numbers, we only get a ratio of ~5 (for both junior and senior hires), i.e. however valuable people think a role being filled is, they think the value of getting a 'hire-level' person to the community is approximately 5x this. 

This seems more in line with the number of additional roles that we might imagine a typical hire goes onto after being hired for their first role. That said, people might also have been imagining (i) that people's value produced increases (perhaps dramatically) after their first role, (ii) that people create value for the community outside the roles they're hired to. 

Thanks for the comment Jessica! This makes sense. I have a few thoughts about this:

  • More time for people to answer, and in particular to reflect, sounds like it could have been useful (though I wasn't at the event, so I can't comment on the tradeoffs here).
  • My impression is that the difficulty of the survey is mostly due to the inherent difficulty of the questions we were asked to elicit judgements about (either/both because the questions were substantively difficult and required a lot of information/reflection- e.g. what is the optimal growth rate for EA- or because they're very conceptually difficult/counterintuitive- e.g. how much value do you assign to x relative to y controlling for the value of x's converting into y's), and less because of the operationalization of the questions themselves (see the confusion about earlier iterations of the questions).
    • One possible response to this, which was mentioned in feedback, is that it could be valuable to pose these questions to dedicated working groups, who devote extensive amounts of time to deliberating on them. Fwiw, this sounds like a very useful (though very costly) initiative to me. It would also have the downside of limiting input to an even smaller subset of the community: so perhaps ideally one would want to pose these questions to a dedicated group, presenting their findings to the wider MCF audiences, and then ask the MCF audience for their take after hearing the working group's findings. Of course, this would take much more time from everyone, so it wouldn't be valuable.
    • Another possible response is to just try to elicit much simpler judgements. For example, rather than trying to actually get a quantitative estimate of "how many resources do you think think each cause should get?", we could just ask "Do you think x should get more/less?" I think the devil is in the details here, and it would work better for some questions than others e.g. in some cases, merely knowing whether people think a cause should get more/less would not be action-guiding for decisionmakers, but in other cases it would (we're entirely dependent on what decisionmakers tell us they want to elicit here, since I see our role as designing questions to elicit the judgements we're asked for, not deciding what judgements we should try to elicit). 

Hey Manuel,

I think the public posts should start coming out pretty soon (within the next couple of weeks). 

That said I would strongly encourage movement builders and other decision-makers to reach out to us directly and request particular results when they are relevant to your work. We can often produce and share custom analyses within a day (much faster than a polished public post).

Load more