Hey this is super cool!
I think this could be a great intervention, but I disagree on the RCT front. I strongly believe that in the medium to long-term, this intervention should, and will only go to scale after high quality RCTs.
Yours and RP's reasons for not doing it aren't compelling to me. I'm almost a bit confused.. This might actually be one of the easier RCTs to do in the development world. If I was a funder, I would be asking why you aren't in the middle of a big RCT at the moment, or at least why you aren't getting started on one. I agree with the funders.
"Funders often tell us to come back with an RCT, which we estimate would need around 72 hospitals across three countries, US$5 million and three to four years." If this is what's needed, then yeah do it. You can get the 5 million - have you asked GiveWell? But I'm skeptical you really need 72 hospitals. Mortality rates are high enough in massive hospitals I would imagine if you are in national referral hospitals in capital cities, or similar, then this number would drastically drop. Maybe you are considering a bunch of smaller hospitals?
From your article
"conventional RCT is hard to run for this intervention. It can't be blinded, because health workers know when monitors are on the ward. It has to randomise whole hospitals, because the central patient overview and the positive impact on workload affects care for every child on the ward, not only those attached to a monitor. Death is a relatively rare outcome, and baseline neonatal mortality across our sites ranges from 4.3% to 27.8%, which makes a trial harder to power. Ministries are also reluctant to keep hospitals in a control arm for years while neighbouring hospitals have monitoring. So is an RCT needed before scaling? If the effect is real, waiting three to four years for trial results means children in hospitals that could have had monitoring go without it. Last but not least it is important to emphasize that continuous monitoring is already considered standard of care in most parts of the world, so perhaps it is less about if it works but rather to what extent it should be prioritized amidst many other priorities and gaps.
From RP
"Note that a traditional randomized controlled trial may be genuinely hard to run in this context: a monitoring system cannot be disguised as a placebo, and it may change how the whole ward operates. For example, staff know when the monitors are present, and the intervention is bundled with training and supervision. The observed effect size therefore reflects both the device plus the attention surrounding it (i.e., the Hawthorne effect[5]),
1. Your current evidence is fairly weak, and low on the evidence ladder. Before and after studies are notorious for being unreliable because there are so many other factors involved. Hospitals are always improving their systems and there are many reasons why mortality could drop. Seeing consistency between studies is encouraging but evidence needs to be better.
2. Blinding is not very important - the staff seeing the monitors is part of the intervention no? I'm not sure why RP and you think this is a problem? If it changes how the whole ward operates that is part of the benefit of the intervention. In fact that might be where you main benefit is! I'm confused about this comment by RP here and not sure what point they are making.... The training and supervision front could be a big deal though, especially when you believe your main impact to be ward spillovers. That's why I think training and supervision can't really last more than a few months, then you need to largely leave the hospital be with the monitors with minimal supervision and see if mortality reductions are sustained over 2-3 years. This isn't that long an RCT!
4. Randomising whole hospitals isn't an ethical issue. Ministries will not refuse to allow hospitals being in a control arm (they are getting free monitors) and the evidence isn't nearly good enough yet to argue an RCT is unethical. If the difference in mortality between hospitals is big after year 2 say, the study will be triggered to stop for ethical reasons anyway.
5. The range of mortality rates doesn't make powering a study harder I don't think? (might be missing something) Its low mortality rates and low sample sizes that might be the issue here. But there are lots of hospitals where multiple kids die every week (I have one 500 meters from my home) If you are in the busiest hospitals where 150+ kids die every year, then I would think you wouldn't need neadly that amount (I might Claude this later). If you need to do a multi-country RCT so be it, that seems likely needed.
6. The urgency argument seems unrealistic. Because of the low level of evidence intervention isn't "obviously" better than other things we could spend money on. I don't think your case is nearly strong enough here to make a "bypass the RCT" argument. I think actually even framing it like this could make people lose confidence in the intervention. It feels more like a sales pitch than a good argument.
7. What is considered "Standard of care" in rich countries is close irrelevant in healthcare interventions in low-income countries. I don't think it should even be a factor really. There are 100 "standard of care" things you could introduce to a pediatric ward in Malawi (Better antibiotics, Dr. nos, nursing ratios, frequency checks, infection control, blood cultures, blood gases etc.). This doesn't mean you don't study their benefit in a low-income setting.
As an unrelated point, I feel like the presentation of the intervention here might be too branded. The intervention is a monitoring device, yet you are at pain to emphasise how your intervention is different from other devices. If your plan is to scale through government, its the monitoring system that matters not the branding. It might be better even to brand machines as "Ministry of Health" machines and think of yourself as a support org as much as an innovation org? This is a tricky one though I know.
I absoutely love this intervention - I'd be keen to get on a call and chat about it even if you might be. Sadly someone ffrom your org was pegged to meet me at EAG New York, but I can't go because of travel restrictions due to ebola :(
We got an op ed published in a major Australian newspaper today in response to the recent wave of media about EA: https://www.afr.com/technology/trump-calls-effective-altruism-a-dangerous-ideology-it-s-not-20261007-p6135s
This comment section felt a bit brigaded here, which I don't love.
I'm sympathetic to debating strategic disagreements in the animal welfare movement, and appreciate you bringing these ideas up and pushing for what you believe is the right approach.
There were 2 things that stood out to me:
We need to go beyond Peter Singer’s utilitarian approach, which has had a limiting impact on campaigns giving preference to concerns about animal suffering and welfare reforms. What is needed are a deontological philosophy and a holistic, ecological systems approach.
I don't think utilitarianism gives a preference to welfare reforms — it's that welfare reforms have a strong track record of being effective, while these other approaches don't. Peter Singer himself wholeheartedly embraces animal liberation, literally! If the utilitarians thought that things beyond welfare reforms would work, and were cost-effective, they'd do them. It isn't because of their principled commitment to utilitarianism that they disagree.
Secondly, I think that many of the donors and EAs would stop being EA donors or EAs at all if deontology became the norm. At least speaking for myself, I am not aligned with EA because it is EA, but I'm aligned with EA because it is more aligned with my values (which are very utilitarian). I suspect this is true for many EA-motivated people. If EA changes, it doesn't mean the donors will change their values.
The campaign was inspired by: (1) the “AI Windfall” entering the animal movement
I think an important question for people who aren't excited about new funding entering the space going to welfare reforms is "why are these new donors are interested in funding animal welfare work at all?"
I think the answer is because cage-free campaigns, etc. demonstrated that helping animals at scale was possible, and because welfare-motivated advocates demonstrated that something could be done to help animals effectively.
No animal welfare advocate controls where these donors give — if the movement today decided to pivot to plant-based advocacy and stop doing welfare stuff, it's plausible significantly less money would be donated to helping animals. There isn't a pot of unlimited money controlled by animal advocates — there are donors who have specific values and priorities, and organizations who will make the case to those donors that their work should be funded. The pitch should be to the donors (who actually will decide these things), not to EA as a whole, and has to overcome the fact that you're pitching against the thing that might have gotten many of them excited about animal advocacy in the first place.
I'm currently hiring. Come and help me address the suffering of animals. At Rethink Priorities:
Senior Research Manager – To lead a new Animal Welfare Strategy Team. This team will focus on supporting and accelerating funders and organizations toward achieving accelerating the end of low-welfare farming. The team lead will have to manage a team to do consulting for animal organisations, in-house research projects, and coordinate the Animals Advocacy strategy Forum. Salary: $98,500–$125,000. Deadline: Monday 19 October.
https://careers.rethinkpriorities.org/en/postings/bc1c94f2-31d6-4e04-817c-ab1e7e6c26c9
(Senior) Neglected Animals Program Officer – Working to improve the welfare of terrestrial invertebrate, both farmed and wild. The role will be a mix of research and coordination and involve doing analysis, commissioning and overseeing external contractors, and coordinating with organizations working on shared problems. Salary: $79,000–$98,500. Deadline: Sunday 11 October.
https://careers.rethinkpriorities.org/postings/a7f585b8-7c7f-4748-b9eb-3d836ffc3683
I'm continuing the work on my SPAR project, and I met with a hiring manager at an AI Safety org this week. Here's some interesting things I've learned:
- They use a multiple of tools and processes - a mixture of HR tools, AI and human review to score and review applications and work tests. They stated the process felt "disjointed".
- Work tests are developed for multiple roles are broken down into 2-3 types of tests:
- General - a foundational test that each applicant receives at this stage to map to cultural fit
- Role specific - work problem (based upon junior or senior role type) that is reviewed by human reviewers.
- Work tests are developed by an internal team (not just the hiring manager) and they use archives of past (and high scoring responses) + team to create tests. Scoring (initial screening is done by AI with human review).
- Their biggest frictions in the hiring process:
- Not being able to review in a timely manner completed work tests, and therefore their concern is they are losing high quality candidates to other orgs.
- When the potential tidal increase of more candidates coming, due to expected IPO's, then their own hiring strategy may be to only hire from referrals.
This conversation makes me realize that the hiring process is not only being hindered by processes (disjointed or non-existent) but also that most orgs (even with processes or tools) are not ready for the influx of people that could potentially look to AI Safety orgs as how they can contribute.
Therefore, my two may crux's that I need to explore more (with more discussions with AI Safety hiring managers) are the following:
- Is the hiring process inefficient due to the process (or lack of it) and a Playbook for hiring managers would help?
- How are work tests being written, how are the being scored and is the "reviewer" knowledgeable about the specific role that they can determine if the applicant has not only done this work in the past (and knows what they are talking about) but also is it good work (and not just a basic understanding)?
My ask for help: I'm looking for AI Safety founders or hiring managers who would be willing to conduct a short interview (15 mins) with me to help me understand the issues that you are facing. All information that you share with me will be only shared with my SPAR mentor, and my final published report will only include generialiations (and not specifics to any individual or org) in regards to your internal processes. I can be reached at: [email protected]
Will MacAskill's "Saturation View" write-up describes a view that tries to get around the Mere Addition Paradox and the Repugnant Conclusion by valuing variety.
This fails "Non-Anti-Egalitarianism" i.e. it sometimes places a lower value on a world where the same people have a more equal welfare and also a greater average welfare. He claims that it satisfies a weaker version in which the people don't change their "types" but I think that's not correct.
For his toy example (see screenshot): Call the top-left and bottom-right squares in the 3x3 grid red and blue. Consider two worlds:
The unequal world has total welfare 16 while the equal world has total welfare 18. The cumulative welfare of the lowest-welfare N members is also greater for the equal world for all N from 1 to 9.
However, Toy-Saturationism awards the unequal world a value of 24/5 = 4.8, which is greater than the equal world's value of 232/5 ≈ 4.2.
The choice of as the curve isn't important because we can always construct a similar case in which equalization can result in a lower value.
In reality, I expect saturation views suffer badly from this problem of creating inequality for the sake of the "variety" values they introduce.