New set of evaluations
The Unjournal evaluations of Artificial Intelligence and Economic Growth, by prominent economists Philippe Aghion, Benjamin F. Jones, Charles I. Jones – are up.
You can read these on our new PubPub community space , along with my discussion of the process and the insights and the 'evaluation metrics', and the authors' response. Thanks to the authors for their participation (reward early-adopters who stick their necks out!), and thanks to Philip Trammel and Seth Benzell for detailed and insightful evaluation.
I discussed some of the reasons we 'took on' this paper in an earlier post. The discussion of AI's impact on the economy, what it might look like (in magnitude and in its composition), how to measure and model it, and what conditions lead to "growth explosions", seem especially relevant to recent events and discussion.
"Self-correcting" science?
I'm particularly happy about one outcome here.
If you were a graduate student reading the paper, or were a professional delving into the economics literature, and had seen the last step of the equations pasted below (from the originally published paper/chapter), what would you think?
The final step in fact contains an error; the claimed implication does not follow.
From my discussion:
... we rarely see referees and colleagues actually reading and checking the math and proofs in their peers’ papers. Here Phil Trammel did so and spotted an error in a proof of one of the central results of the paper (the ‘singularity’ in Example 3). ... The authors have acknowledged this error ... confirmed the revised proof, and link a marked up version on their page. This is ‘self-correcting research’, and it’s great!
Even though the same result was preserved, I believe this provides a valuable service.
Readers of the paper who saw the incorrect proof (particularly students) might be deeply confused. They might think ‘Can I trust this papers’ other statements?’ ‘Am I deeply misunderstanding something here? Am I not suited for this work?’ Personally, this happened to me a lot in graduate school; at least some of the time it may have been because of errors and typos in the paper. I suspect many math-driven paper also contain flaws which are never spotted, and these sometimes may affect the substantive results (unlike in the present case).
By the way, the marked up 'corrected' paper is here, and the corrected proof is here. (Caveat: Philip and the authors have agreed on the revised corrected proof, it might benefit from an independent verification.)
New (additional) platform: PubPub
We are trying out the PubPub platform. We are still maintaining our Sciety page, and we aim to import the content from one to the other, for greater visibility. Some immediate benefits of PubPub...
- It lets us assign 'digital object identifiers' (DOIs) for each evaluation, response, and summary. It puts these and the works referenced into the 'CrossRef' database.
- Jointly, this should (hopefully) enable indexing in Google Scholar and other academic search engines,
- And 'bibliometrics' (citation counts etc.0
- It seems to enable evaluations of work hosted anywhere that has a DOI (published, preprints, etc.)
- It's versatile and full-featured, enabling input from and output from a range of formats, as well as community input and discussion
- It's funded by a non-profit and seems fairly mission-aligned
More coming soon, updates
The Unjournal has several more impactful papers evaluated and being evaluated, which we hope to post soon. For a sense of what's coming, see our 'Direct Evaluation track' focusing on NBER working papers.
Some other updates:
- We are pursuing collaborations with replication and robustness initiatives such as the "Institute for Replication" and repliCATS
- We are now 'fiscally sponsored' by the Open Collective Foundation; see our page HERE. (Note, this is an administrative thing, it's not a source of funding)
You can follow our latest updates on our gitbook page 'latest updates' HERE. I'll try to maintain this, and we are looking to build an email list thing soon.