Notes on how prizes may fail and how to reduce the risk of them failing

Thank you for writing this up - I’ve wanted to do the same for a while! I think the only thing I see missing is that prizes can raise the salience of some concept or nuance, and therefore serve as a coordination mechanism in more ways than you list (e.g., say that we want more assessments of long-term interventions using the framework from WWOTF of significance - durability - contingency, then a prize for those assessments would also help signal boost the framework)

Another downside is that it eats up quite a lot of time. E.g. if we take the Cause Exploration Prize and assume:

• there are 143 entries (the tag shows 144 posts with that tag on the Forum, one of which introduces the prize)
• an average entry takes ~27h to research and write up (90% CI 15-50h)
• an average entry takes ~1.4h to judge (90% CI 0.5-4h, but maybe I'm wildly underestimating this?)

then we get ~2 FTE years spent (90% CI 1.2-3.6 years). That's quite a lot of labour spent by engaged and talented EAs (and ppl adjacent to EA)!

(Caveats: Those assumptions are only off-the-cuff guesses. It's not clear to me what the counterfactual is, but presumably some of these hours wouldn't have been spent doing productive-for-EA work. Also, I'm not sure whether, had you hired a person to think of new cause areas for 2 years, they would've done as well, and at any rate it would've taken them 2 years!)

Edit: To be clear, I'm not saying the Prize isn't worth it. I just wanted to point out a cost that may to some degree be hidden when the org that runs a contest isn't the one doing most of the labour.

Thanks for this! Anecdotally, there are several prizes where I've fleetingly thought 'hmm, maybe I'll try and write something for that...', but then....not. In my case this is a combination between not being (able to be?) risk-neutral about personal finance and maybe feeling obligations to others more keenly than the pull to do something cool/fun. This might me more of a problem with me than with prizes, but I just wanted to add myself to the anecdotal pile of people who (it seems) don't seem to 'respond well' to prizes as an incentive. 

One interesting debate would be: what’s the optimal % of funding that should go to prizes? Which parameters would allow us to determine this? One can imagine that the % should be higher in communities that are struggling more to hire enough, or where research agendas are unclear so more coordination is needed, but should be lower in communities with people with low savings, or where the funders have capacity to diversify risks.

One additional consideration is that the coordination benefits from prizes (in raising the salience of memes or the status of the winners) comes at an attention cost, so a large number of prizes may cannibalize on our “common knowledge budget” (if there is a limit to how much common knowledge we can generate)

Great post. I wish there was a way other people could contribute to it and flesh it out.

Let me know what you think about this format and I may do it more. 

I like this format, especially if the counterfactual is no post!

I like this style of post with high density of information. I'm not sure a longer post would add additional value, especially after taking into account the additional time to write/read.

I would add that I'm in favour of contests with as vague criteria as possible. I'd rather have a contest that says "$X for the best forum post!" than one that says "$X for the best forum post that fits criteria ABC". I think generally you want to incentivise people to work on and write down what they think are their most interesting ideas, as judged by themselves. Model them as having unexpectedly valuable insights in their brains, and you're using prizes to extract it. 

But when you incentivise them to compromise what they are optimising for in order to fit your criteria, the results are likely to disappoint.^[1]

1. ^{^}

   I think under some conditions, on a toy model 🦋, the extent to which the results disappoint you will be exponentially proportional to the extent you make them conjunctively-compromise what they optimise for.