It could also be that the way innovation contests are implemented in real life is severely suboptimal :)
After all, the FCC only started using spectrum auctions in the mid 90s, while academic work on auction theory/mech design is significantly older. We have also only recently started using insights from game theory in school choice, even though Gale and Shapley published their paper on the deferred acceptance algorithm in 1962.
I have really enjoyed reading this report. Thanks for producing this public good! Here are some thoughts I had while reading it.
First, when should a sponsor use a contest and not some other mechanism? I agree with the report that a contest is e a good idea when marketplace incentives are weak, and the path to the innovation is not clear. I'd add that even in those circumstances, the sponsor could engage in contract research or provide a grant to a researcher. I would guess that contests are preferable to those alternatives when identifying suitable candidates for grants/research is difficult or moral hazard problems are particularly salient.
Second, what are the possible designs of innovation contests that could be used?
One important design decision is whether to have a fixed deadline when the participants are evaluated and the winners are determined (as in the 2005 DARPA Grand Challenge) or to have a fixed objective so that the first participant to achieve the objective wins (as in Clay Institute's Millenium Prize Problems or the original Longitude Prize). The advantage of a fixed deadline is that the goal specification can be more vague, but runs the risk that no participant will have a satisfactory result. Fixed objective prizes can run (and provide incentives) for a long time, but require a very clear and objective goal specification.
With fixed deadline contests, the designer can also choose whether to have elimination rounds (like in NRG Cosia X Prize contest). I am not familiar with any work analyzing the effect of elimination rounds.
Whether to give one or multiple final prizes (which was discussed in the report) and whether to give interim prizes while the contest is ongoing (as was the case in the Netflix Prize contest).
What information to release to the participants during the contest, especially about their current standing relative to other participants? For example, whether to have a leaderboard or not. (Lemus and Marshall have some interesting results on this.)
Third, recognition prizes are cool, but I don't see how they can be used to generate research aimed at solving a specific problem. As such, they can (likely?) only provide broad incentives to engage in research in some field.
Fourth, not all innovation contests need to generate a lot of publicity. I am actually hoping to see more of smaller, targeted innovation contests that aim to solve real but unglamorous problems people face (in particular, those in less developed countries). With such problems, monetary rewards may be more important than they were in, say, the Ansari Prize.
Here is a very quick and incomplete list. Some papers are less directly related, e.g., “Contests for status” is not about innovation contests per se, but if we know that a large appeal from inducement prizes comes from status (as the report also points out), we might still learn something from the paper.
Edit: added *s to indicate papers more directly related.
*Taylor, C. R. (1995). Digging for golden carrots: An analysis of research tournaments. The American Economic Review, 872-890.
Moldovanu, B., & Sela, A. (2001). The optimal allocation of prizes in contests. American Economic Review, 91(3), 542-558.
*Che, Y. K., & Gale, I. (2003). Optimal design of research contests. American Economic Review, 93(3), 646-671.
Yildirim, H. (2005). Contests with multiple rounds. Games and Economic Behavior, 51(1), 213-227.
Moldovanu, B., & Sela, A. (2006). Contest architecture. Journal of Economic Theory, 126(1), 70-96.
Moldovanu, B., Sela, A., & Shi, X. (2007). Contests for status. Journal of political Economy, 115(2), 338-363.
*Ding, W., & Wolfstetter, E. G. (2011). Prizes and lemons: procurement of innovation under imperfect commitment. The RAND Journal of Economics, 42(4), 664-680.
*Fu, Q., Lu, J., & Lu, Y. (2012). Incentivizing R&D: Prize or subsidies?. International Journal of Industrial Organization, 30(1), 67-79.
Halac, M., Kartik, N., & Liu, Q. (2017). Contests for experimentation. Journal of Political Economy, 125(5), 1523-1569.
*Letina, I., & Schmutzler, A. (2019). Inducing variety: A theory of innovation contests. International economic review, 60(4), 1757-1780.
*Gross, D. P. (2020). Creativity under fire: The effects of competition on creative production. Review of Economics and Statistics, 102(3), 583-599.
*Benkert, J. M., & Letina, I. (2020). Designing dynamic research contests. American economic journal: Microeconomics, 12(4), 270-89.
*Che, Y. K., Iossa, E., & Rey, P. (2021). Prizes versus contracts as incentives for innovation. The Review of Economic Studies, 88(5), 2149-2178.
*Lemus, J., & Marshall, G. (2021). Dynamic tournament design: Evidence from prediction contests. Journal of Political Economy, 129(2), 383-420.
*Erkal, N., & Xiao, J. (2021). Scarcity of ideas and optimal prizes in innovation contests. Available at SSRN.
*Chen, B., Chen, B., & Knyazev, D. (2022). Information disclosure in dynamic research contests. The RAND Journal of Economics, 53(1), 113-137.
This seems like a great effort, and I do not want to be overly critical. I would just like to point out that the report overlooks quite a few papers from economics (as distinct from management), which look at strategic behavior in innovation contests, optimal design of innovation contests, and empirical evaluations of existing contests. Since the report states in the intro that one of the main goals is the review of the existing literature, this seemed like an important point.
It could also be that the way innovation contests are implemented in real life is severely suboptimal :)
After all, the FCC only started using spectrum auctions in the mid 90s, while academic work on auction theory/mech design is significantly older. We have also only recently started using insights from game theory in school choice, even though Gale and Shapley published their paper on the deferred acceptance algorithm in 1962.
I have really enjoyed reading this report. Thanks for producing this public good! Here are some thoughts I had while reading it.
First, when should a sponsor use a contest and not some other mechanism? I agree with the report that a contest is e a good idea when marketplace incentives are weak, and the path to the innovation is not clear. I'd add that even in those circumstances, the sponsor could engage in contract research or provide a grant to a researcher. I would guess that contests are preferable to those alternatives when identifying suitable candidates for grants/research is difficult or moral hazard problems are particularly salient.
Second, what are the possible designs of innovation contests that could be used?
Third, recognition prizes are cool, but I don't see how they can be used to generate research aimed at solving a specific problem. As such, they can (likely?) only provide broad incentives to engage in research in some field.
Fourth, not all innovation contests need to generate a lot of publicity. I am actually hoping to see more of smaller, targeted innovation contests that aim to solve real but unglamorous problems people face (in particular, those in less developed countries). With such problems, monetary rewards may be more important than they were in, say, the Ansari Prize.
Here is a very quick and incomplete list. Some papers are less directly related, e.g., “Contests for status” is not about innovation contests per se, but if we know that a large appeal from inducement prizes comes from status (as the report also points out), we might still learn something from the paper.
Edit: added *s to indicate papers more directly related.
*Taylor, C. R. (1995). Digging for golden carrots: An analysis of research tournaments. The American Economic Review, 872-890.
Moldovanu, B., & Sela, A. (2001). The optimal allocation of prizes in contests. American Economic Review, 91(3), 542-558.
*Che, Y. K., & Gale, I. (2003). Optimal design of research contests. American Economic Review, 93(3), 646-671.
Yildirim, H. (2005). Contests with multiple rounds. Games and Economic Behavior, 51(1), 213-227.
Moldovanu, B., & Sela, A. (2006). Contest architecture. Journal of Economic Theory, 126(1), 70-96.
Moldovanu, B., Sela, A., & Shi, X. (2007). Contests for status. Journal of political Economy, 115(2), 338-363.
*Ding, W., & Wolfstetter, E. G. (2011). Prizes and lemons: procurement of innovation under imperfect commitment. The RAND Journal of Economics, 42(4), 664-680.
*Fu, Q., Lu, J., & Lu, Y. (2012). Incentivizing R&D: Prize or subsidies?. International Journal of Industrial Organization, 30(1), 67-79.
Halac, M., Kartik, N., & Liu, Q. (2017). Contests for experimentation. Journal of Political Economy, 125(5), 1523-1569.
*Letina, I., & Schmutzler, A. (2019). Inducing variety: A theory of innovation contests. International economic review, 60(4), 1757-1780.
*Gross, D. P. (2020). Creativity under fire: The effects of competition on creative production. Review of Economics and Statistics, 102(3), 583-599.
*Benkert, J. M., & Letina, I. (2020). Designing dynamic research contests. American economic journal: Microeconomics, 12(4), 270-89.
*Che, Y. K., Iossa, E., & Rey, P. (2021). Prizes versus contracts as incentives for innovation. The Review of Economic Studies, 88(5), 2149-2178.
*Lemus, J., & Marshall, G. (2021). Dynamic tournament design: Evidence from prediction contests. Journal of Political Economy, 129(2), 383-420.
*Erkal, N., & Xiao, J. (2021). Scarcity of ideas and optimal prizes in innovation contests. Available at SSRN.
*Chen, B., Chen, B., & Knyazev, D. (2022). Information disclosure in dynamic research contests. The RAND Journal of Economics, 53(1), 113-137.
This seems like a great effort, and I do not want to be overly critical. I would just like to point out that the report overlooks quite a few papers from economics (as distinct from management), which look at strategic behavior in innovation contests, optimal design of innovation contests, and empirical evaluations of existing contests. Since the report states in the intro that one of the main goals is the review of the existing literature, this seemed like an important point.