EA Norway has shifted from in-person to digital general assemblies since COVID. This change has sparked some ongoing debate.
Benefits of in-person assemblies:
More informal networking and conversation
Better discussion environment
More enjoyable experience (digital meeting fatigue is real)
Previously combined with weekend conferences featuring talks and group discussions
Benefits of digital assemblies:
Easier attendance
Especially for members with families
Especially for people not living in Oslo, the capitol
Lower costs (minor)
Lower bar of entry for new members
EA Norway now also maintains an annual in-person gathering, essentially a mini-EAGx for Norway, were we plan to increasingly focus on organizational strategic planning to better capture some of the benefits of an in-person assembly.
Thanks for sharing this update. I appreciate the transparency and your engagement with the broader community!
I have a few questions about this strategic pivot:
On organizational structure: Did you consider alternative models that would preserve 80,000 Hours' established reputation as a more "neutral" career advisor while pursuing this AI-focused direction? For example, creating a separate brand or group dedicated to AI careers while maintaining the broader 80K platform for other cause areas? This might help avoid potential confusion where users encounter both your legacy content presenting multiple cause areas and your new AI-centric approach.
On the EA pathway: I'm curious about how this shift might affect the "EA funnel" - where people typically enter effective altruism through more intuitive cause areas like global health or animal welfare before gradually engaging with longtermist ideas like AI safety. By positioning 80,000 Hours primarily as an AI-focused organization, are you concerned this might make it harder for newcomers to find their way into the community if AI risk arguments initially seem abstract or speculative to them?
On reputational considerations: Have you weighed the potential reputational risks if AI development follows a more moderate trajectory than anticipated? If we see AI plateau at impressive but clearly non-transformative capabilities, this strategic all-in approach could affect 80,000 Hours' credibility for years to come. The past decade of 80K's work as a cause-diverse advisor has created tremendous value - might a spinoff organization for AI-specific work better preserve that accumulated trust while still allowing you to pursue what you see as the highest-impact path?
The slides for GiveWell’s 2020 analysis of AMF are stellar, hopefully we can draw from them at Gi Effektivt! I particularly liked the slides that draws directly from the CEAs.
From page 18: "Between 2000 and 2019, case incidence in the WHO African Region decreased from 370 to 226 per 1000 population at risk, but increased to 232 per 1000 population at risk in 2020, mainly because of disruptions to services during the COVID-19 pandemic. In 2022, case incidence declined to 223 per 1000 population at risk."
This is an argument for the effectiveness of existing interventions.
Funding to malaria prevention has stalled since 2010.[1] It's misleading to point to GiveWell funding alone. I don't think it's particularly surprising that given ~constant funding the progress has slowed. As noted the nets last only a couple of years, and presumably you get diminishing marginal returns when scaling up, as always. I'm not sure exactly how they are counting here, as "other funders" is suspiciously small, but the general point still stands, that the largest donors have stalled over the last decade. WHO estimates for the counterfactual[2] show that just keeping the deaths constant is an achievement in itself (although the counterfactual is of course more uncertain). I think this comes down to population growth and a little bit of climate change, but I haven't looked into it deeply.
I think there is an argument to be made here, that as the world shifted priorities with the SDGs and funding for the "old" efforts stalled, we unfortunately got a unusually good opportunity in malaria prevention in terms of marginal impact. While just keeping up the pressure doesn't yield equally spectacular sounding results as the initial ramp up, it's still likely saving thousands of lives.
World Health Organization. (2023). Funding for malaria control and elimination, 2000–2022, by channel (constant 2022 US$) [Figure 6.4]. In World Malaria Report 2023 (p. 49). WHO.
Yes, this makes sense if I understand you correctly. If we set the effect size to 0 for all the dropouts, while having reasonable grounds for thinking it might be slightly positive, this would lead to underestimate top-line cost effectiveness.
I'm mostly reacting to the choice of presenting the results of the completer subgroup which might be conflated with all participants in the program. Even the OP themselves seem to mix this up in the text.
Context: To offer a few points of comparison, two studies of therapy-driven programs found that 46% and 57.5% of participants experienced reductions of 50% or more, compared to our result of 72%. For the original version of Step-by-Step, it was 37.1%. There was an average PHQ-9 reduction of 6 points compared to our result of 10 points.
As far as I can tell, they are talking about completers in this paragraph, not participants. @RachelAbbott could you clarify this?
When reading the introduction again I think it's pretty balanced now (possibly because it was updated in response to the concerns). Again, thank you for being so receptive to feedback @RachelAbbott!
I hope this is not what is happening. It's at best naive. This assumes no issues will crop up during scaling, that "fixed" costs are indeed fixed (they rarely are) and that the marginal cost per treatment will fall (this is a reasonable first approximation, but it's by no means guaranteed). A maximally optimistic estimate IMO. I don't think one should claim future improvements in cost effectiveness when there are so many incredibly uncertain parameters in play.
My concrete suggestion would be to rather write something like: "We hope to reach 10 000 participants next year with our current infrastructure, which might further improve our cost-effectiveness."
Thanks for this thorough and thoughtful response John!
I think most of this makes sense. I agree that if you are using an evidence based-intervention, it might not make sense to increase the cost by adding a control group. I would for instance not think of this as a big issue for bednet distribution in an area broadly similar to other areas bednet distribution works. Given that in this case they are simply implementing a programme from WHO with two positive RCTs (which I have not read), it seems reasonable to do an uncontrolled pilot.
I pushed back a little in a comment from you further down, but I think this point largely addresses my concerns there.
With regards to your explanations for why people drop out, I would argue that at least 1,2 and 3 are in fact because of the ineffectiveness of the intervention, but it's mostly a semantic discussion.
The two RCTs cited seem to be about displaced Syrians, which makes me uncomfortable straightforwardly assuming it will transfer to the context in India. I would also add that there is a big difference between the evidence base for ITN distribution compared to this intervention. I look forward to seeing what the results are in the future!
EA Norway has shifted from in-person to digital general assemblies since COVID. This change has sparked some ongoing debate.
Benefits of in-person assemblies:
Benefits of digital assemblies:
EA Norway now also maintains an annual in-person gathering, essentially a mini-EAGx for Norway, were we plan to increasingly focus on organizational strategic planning to better capture some of the benefits of an in-person assembly.
I'm not the expert on effective altruism. I don't identify with that terminology. My impression is that it's a bit of an outdated term.
Thanks for sharing this update. I appreciate the transparency and your engagement with the broader community!
I have a few questions about this strategic pivot:
On organizational structure: Did you consider alternative models that would preserve 80,000 Hours' established reputation as a more "neutral" career advisor while pursuing this AI-focused direction? For example, creating a separate brand or group dedicated to AI careers while maintaining the broader 80K platform for other cause areas? This might help avoid potential confusion where users encounter both your legacy content presenting multiple cause areas and your new AI-centric approach.
On the EA pathway: I'm curious about how this shift might affect the "EA funnel" - where people typically enter effective altruism through more intuitive cause areas like global health or animal welfare before gradually engaging with longtermist ideas like AI safety. By positioning 80,000 Hours primarily as an AI-focused organization, are you concerned this might make it harder for newcomers to find their way into the community if AI risk arguments initially seem abstract or speculative to them?
On reputational considerations: Have you weighed the potential reputational risks if AI development follows a more moderate trajectory than anticipated? If we see AI plateau at impressive but clearly non-transformative capabilities, this strategic all-in approach could affect 80,000 Hours' credibility for years to come. The past decade of 80K's work as a cause-diverse advisor has created tremendous value - might a spinoff organization for AI-specific work better preserve that accumulated trust while still allowing you to pursue what you see as the highest-impact path?
The slides for GiveWell’s 2020 analysis of AMF are stellar, hopefully we can draw from them at Gi Effektivt! I particularly liked the slides that draws directly from the CEAs.
WHO claims the bump is due to covid-19 disruptions in world malaria report 2023.
From page 18:
"Between 2000 and 2019, case incidence in the WHO African Region decreased from 370 to 226 per 1000 population at risk, but increased to 232 per 1000 population at risk in 2020, mainly because of disruptions to services during the COVID-19 pandemic. In 2022, case incidence declined to 223 per 1000 population at risk."
This is an argument for the effectiveness of existing interventions.
Funding to malaria prevention has stalled since 2010.[1] It's misleading to point to GiveWell funding alone. I don't think it's particularly surprising that given ~constant funding the progress has slowed. As noted the nets last only a couple of years, and presumably you get diminishing marginal returns when scaling up, as always. I'm not sure exactly how they are counting here, as "other funders" is suspiciously small, but the general point still stands, that the largest donors have stalled over the last decade. WHO estimates for the counterfactual[2] show that just keeping the deaths constant is an achievement in itself (although the counterfactual is of course more uncertain). I think this comes down to population growth and a little bit of climate change, but I haven't looked into it deeply.
I think there is an argument to be made here, that as the world shifted priorities with the SDGs and funding for the "old" efforts stalled, we unfortunately got a unusually good opportunity in malaria prevention in terms of marginal impact. While just keeping up the pressure doesn't yield equally spectacular sounding results as the initial ramp up, it's still likely saving thousands of lives.
World Health Organization. (2023). Funding for malaria control and elimination, 2000–2022, by channel (constant 2022 US$) [Figure 6.4]. In World Malaria Report 2023 (p. 49). WHO.
World Health Organization. (2023). WHO methods for estimating cases and deaths averted. In World Malaria Report 2023 (p. 123), Annex 1. WHO.
Yes, this makes sense if I understand you correctly. If we set the effect size to 0 for all the dropouts, while having reasonable grounds for thinking it might be slightly positive, this would lead to underestimate top-line cost effectiveness.
I'm mostly reacting to the choice of presenting the results of the completer subgroup which might be conflated with all participants in the program. Even the OP themselves seem to mix this up in the text.
As far as I can tell, they are talking about completers in this paragraph, not participants. @RachelAbbott could you clarify this?
When reading the introduction again I think it's pretty balanced now (possibly because it was updated in response to the concerns). Again, thank you for being so receptive to feedback @RachelAbbott!
Very interesting, thanks for highlighting this!
I hope this is not what is happening. It's at best naive. This assumes no issues will crop up during scaling, that "fixed" costs are indeed fixed (they rarely are) and that the marginal cost per treatment will fall (this is a reasonable first approximation, but it's by no means guaranteed). A maximally optimistic estimate IMO. I don't think one should claim future improvements in cost effectiveness when there are so many incredibly uncertain parameters in play.
My concrete suggestion would be to rather write something like: "We hope to reach 10 000 participants next year with our current infrastructure, which might further improve our cost-effectiveness."
Thanks for this thorough and thoughtful response John!
I think most of this makes sense. I agree that if you are using an evidence based-intervention, it might not make sense to increase the cost by adding a control group. I would for instance not think of this as a big issue for bednet distribution in an area broadly similar to other areas bednet distribution works. Given that in this case they are simply implementing a programme from WHO with two positive RCTs (which I have not read), it seems reasonable to do an uncontrolled pilot.
I pushed back a little in a comment from you further down, but I think this point largely addresses my concerns there.
With regards to your explanations for why people drop out, I would argue that at least 1,2 and 3 are in fact because of the ineffectiveness of the intervention, but it's mostly a semantic discussion.
The two RCTs cited seem to be about displaced Syrians, which makes me uncomfortable straightforwardly assuming it will transfer to the context in India. I would also add that there is a big difference between the evidence base for ITN distribution compared to this intervention. I look forward to seeing what the results are in the future!