I'm posting this in preparation for Draft Amnesty Week (Feb 24- March 2), but please also use this thread for posts you don't plan to write for Draft Amnesty. The last time I posted this question, there were some great responses.
If you have multiple ideas, I'd recommend putting them in different answers, so that people can respond to them separately.
It would be great to see:
- Both spur-of-the-moment vague ideas, and further along considered ideas. If you're in that latter camp, you could even share a google doc for feedback on an outline.
- Commenters signalling with Reactions and upvotes the content that they'd like to see written.
- Commenters responding with helpful resources or suggestions.
- Commenters proposing Dialogues with authors who suggest similar ideas, or which they have an interesting disagreement with (Draft Amnesty Week might be a great time for scrappy/ unedited dialogues).
Draft Amnesty Week
If the responses here encourage you to develop one of your ideas, Draft Amnesty Week (February 24- March 2) might be a great time to post it. Posts tagged "Draft Amnesty Week" don't have to be thoroughly thought through or even fully drafted. Bullet points and missing sections are allowed. You can have a lower bar for posting.
I would welcome a blog post about RCTs, and if you decide to write one, I hope you consider the perspective below.
As far as I can tell ~0% of nonprofits are interested in rigorously studying their programs in any way, RCTs or otherwise, and I can't help but suspect that this is largely because mostly when we do run RCTs we find that these cherished programs have ~no effect. It's not at all surprising to me that most charities that conduct RCTs feel pressured to do so by donors; but on the other hand basically all charity activities ultimately flow from donor preferences, because donors are the ones with most of the power.
Living Goods is one interesting example, where they ran an RCT because a donor demanded it, got an unexpected (positive) result, and basically pivoted the whole charity based on that. I view that as a success story.
I am certainly not claiming that RCTs are appropriate for all kinds of programs, or some kind of silver bullet. It's more like, if you ask charities "would you like more or less accountability for results", the answer is almost always going to be, "less, thanks".