A multi-disciplinary perspective on emergent and future innovations in peer review [version 3; peer review: 2 approved]

Tennant, Jonathan P.; Dugan, Jonathan M.; Graziotin, Daniel; Jacques, Damien C.; Waldner, Fran�ois; Mietchen, Daniel; Elkhatib, Yehia; Collister, Lauren B.; Pikas, Christina K.; Crick, Tom; Masuzzo, Paola; Caravaggi, Anthony; Berg, Devin R.; Niemeyer, Kyle E.; Ross-Hellauer, Tony; Mannheimer, Sara; Rigling, Lillian; Katz, Daniel S.; Tzovaras, Bastian Greshake; Pacheco-Mendoza, Josmel; Fatima, Nazeefa; Poblet, Marta; Isaakidis, Marios; Irawan, Dasapta Erwin; Renaut, S�bastien; Madan, Christopher R.; Matthias, Lisa; Kj�r, Jesper N�rgaard; O'Donnell, Daniel Paul; Neylon, Cameron; Kearns, Sarah; Selvaraju, Manojkumar; Colomb, Julien

Abstract

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

Citation

Tennant, J. P., Dugan, J. M., Graziotin, D., Jacques, D. C., Waldner, F., Mietchen, D., Elkhatib, Y., Collister, L. B., Pikas, C. K., Crick, T., Masuzzo, P., Caravaggi, A., Berg, D. R., Niemeyer, K. E., Ross-Hellauer, T., Mannheimer, S., Rigling, L., Katz, D. S., Tzovaras, B. G., Pacheco-Mendoza, J., …Colomb, J. (2017). A multi-disciplinary perspective on emergent and future innovations in peer review [version 3; peer review: 2 approved]. F1000Research, 6, Article 1151. https://doi.org/10.12688/f1000research.12037.3

• Journal Article Type: Article
• Acceptance Date: Nov 1, 2017
• Online Publication Date: Nov 29, 2017
• Publication Date: Nov 29, 2017
• Deposit Date: Nov 2, 2020
• Publicly Available Date: Nov 2, 2020
• Journal: F1000Research
• Electronic ISSN: 2046-1402
• Publisher: F1000Research
• Peer Reviewed: Peer Reviewed
• Volume: 6
• Article Number: 1151
• DOI: https://doi.org/10.12688/f1000research.12037.3
• Public URL: https://nottingham-repository.worktribe.com/output/4930504
• Publisher URL: https://f1000research.com/articles/6-1151/v3
• Additional Information: Referee status: Indexed; Referee Report: 10.5256/f1000research.14133.r27485, David Moher, Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada, 10 Nov 2017, version 2, indexed; Referee Report: 10.5256/f1000research.14133.r27486, Virginia Barbour, Queensland University of Technology (QUT), Brisbane, Qld, Australia, 13 Nov 2017, version 2, indexed; Grant Information: TRH was supported by funding from the European Commission H2020 project OpenAIRE2020 (Grant agreement: 643410, Call: H2020-EINFRA-2014-1). The publication costs of this article were funded by Imperial College London. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.; Copyright: This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.