- The "independent evaluations" initiative is now live
- We're sharing a database of ‘[most of] research we are considering, evaluating, or evaluated’ here.
- We're offering a starter bounty: After the first eight quality submissions (or by Jan. 1 2025, whichever comes later), we will award a prize of $500 to the most valuable evaluation (further bounties to be announced given preliminary success.)
Highlighting some key points
The Unjournal focuses on commissioning expert evaluations, guided by an ‘evaluation manager’ and compensating people for their work. However, we see a range of benefits from also encouraging independent evaluation work, thus we're trying out this new initiative.
Who should do these evaluations?
We are particularly looking for people with research training, experience, and expertise in quantitative social science and statistics including cost-benefit modeling and impact evaluation.... But anyone is welcome to give this a try — when in doubt, please go for it.
We are also happy to support collaborations and group evaluations.
We are also keen to work with students and professors to integrate ‘independent evaluation assignments’ (aka ‘learn to do peer reviews’) into research training.
Why should you do an evaluation?
To help ... improving impactful research through journal-independent public evaluation. ... And provide high-quality detailed evaluations to a range of readers/users
We aim to provide feedback to help you become a better researcher and reviewer. We’ll also give prizes for the strongest evaluations. Writing evaluations will help you build a portfolio with The Unjournal, making it more likely we will commission you for paid evaluation work in the future.
Which research?
See “What specific areas do we cover?” .... Especially:
- Research we publicly prioritize... (see the newly-shared database view)
- Research we previously evaluated (see pubpub.unjournal.org)...
- Work other people and organizations suggest as potentially high-impact (see Evaluating Pivotal Questions)
You can also suggest research yourself here (there's another bounty for that) and then do an independent evaluation of it.
What sort of 'evaluation'?
These should somewhat resemble "referee reports for a top journal" but with important differences. We're looking for careful methodological/technical evaluations that focus on research credibility, impact, and usefulness. We ask for discussion as structured ratings: see our 'Academic' form and 'Applied form'.
Discussion
For the full official details, please the link here within our knowledge base. We still may want to adjust this; comments here or in (this Gdoc; we share an older version for inline comments and questions) are still very welcome!
