Head of Impact at ProVeg International, where I lead monitoring, evaluation and learning across six impact areas and 14 country offices. Behavioural scientist by training (PhD) and previously Deputy Director of Experimental Research at Yale, directing experimental behavioural research across the US, UK and India, and Senior Advisor to DEFRA's Strategic Behavioural Insights Team.
My research sits at the intersection of behaviour change and systems change: demand reduction in the illegal wildlife trade, climate communication and activism, and the social diffusion of plant-based diets. ~1,400 citations, h-index 20. Mostly I care about whether interventions actually work, and whether anyone bothered to check.
Re: Value Correlation - this is something we could feasibly actually check rather than just argue about. Conservation and development both have decades of interventions with measured short-run outcomes and later follow-up. I wonder if anyone has tried to estimate the correlation empirically?
Love this!
One suggestion: you mentioned IPV prevalence in Rwanda is ~25%*, but you're only powered to detect a 2pp effect? 2pp reduction with a baseline 25% prevalence is an 8% relative reduction. Anything below that is unlikely to be significant. You're unable to reliably detect a 6 or 7% relative reduction which is above your 5% bar, but would still send you into "pause and explore". I might be better to commit on the confidence interval rather than on significance. Lakens' work on the smallest effect size of interest (SESOI) and equivalence testing could be useful here, since it would let a null result mean you have ruled out effects above some size rather than just failed to find one.
*across a year, while you're measuring across four months. Baseline prevalence is probably smaller in that time period, meaning the minimum detectable effect gets larger in relative terms.
Fully agree with most of what's written here!
>A strong monitoring plan includes a handful of indicators, with the project assigning each a probability of success. These can be scored once a year, with any failures left in. It should take the project around half a day to complete and track things it wants to know anyway.
Just a note on MEL from the inside of an org, it usually takes more than half a day, and that's just for one funder. It gets extremely time-consuming when you have multiple funders each negotiating their own indicator set. What about major donors coordinating on a shared indicator vocabulary?
"75% of organizations say improving MEL is important or urgent, but only 18% would definitely pay for it."
That's a rather upsetting finding. To turn MEL into an ongoing function rather than just a one-off deep dive you need structures in place, people with expertise, tools to collect data etc. It's worth the investment, but funders/orgs definitely have to be willing to put money behind it.
Looking forward to seeing what you produce!
Thanks Ryan! Restitution definitely speaks to me as a personal value (we made this mess, so we have a duty to try to clean it up), but I'm not sure it fully captures why a species in itself is valuable? If the wronged parties are the individual animals whose habitat we cleared, then the duty tracks individuals. Rarity is almost just a proxy for how much harm we have already done.
Diversity clearly has instrumental value (e.g., from an ecosystem services perspective, as an option value), and I think that justifies a great deal of conservation spending. But in the abstract, where does it take us? Ought we be actively breeding new species, and be roughly indifferent between saving one and creating one? That feels wrong to me (again, the intuitionism!)