A Survey of Crowdsourcing in Medical Image Analysis

�rting, Silas N; Doyle, Andrew; van Hilten, Arno; Hirth, Matthias; Inel, Oana; Madan, Christopher R; Mavridis, Dom Panagiotis; Spiers, Helen; Cheplygina, Veronika

Abstract

Rapid advances in image processing capabilities have been seen across many domains, fostered by the application of machine learning algorithms to "big-data". However, within the realm of medical image analysis, advances have been curtailed, in part, due to the limited availability of large-scale, well-annotated datasets. One of the main reasons for this is the high cost often associated with producing large amounts of high-quality meta-data. Recently, there has been growing interest in the application of crowdsourcing for this purpose; a technique that a technique that is well established in a number of disciplines, including astronomy, ecology and meteorology for creating large-scale datasets across a range of disciplines, from computer vision to astrophysics. Despite the growing popularity of this approach, there has not yet been a comprehensive literature review to provide guidance to researchers considering using crowdsourcing methodologies in their own medical imaging analysis. In this survey, we review studies applying crowdsourcing to the analysis of medical images, published prior to July 2018. We identify common approaches and challenges and provide recommendations to researchers implementing crowdsourcing for medical imaging tasks. Finally, we discuss future opportunities for development within this emerging domain.

Citation

Ørting, S. N., Doyle, A., van Hilten, A., Hirth, M., Inel, O., Madan, C. R., …Cheplygina, V. (2020). A Survey of Crowdsourcing in Medical Image Analysis. Human Computation, 7(1), 1-26. https://doi.org/10.15346/hc.v7i1

• Journal Article Type: Article
• Acceptance Date: Aug 20, 2020
• Online Publication Date: Dec 1, 2020
• Publication Date: Oct 1, 2020
• Deposit Date: Sep 15, 2020
• Publicly Available Date: Oct 1, 2020
• Journal: Human Computation
• Print ISSN: 2330-8001
• Publisher: Human Computation Institute
• Peer Reviewed: Peer Reviewed
• Volume: 7
• Issue: 1
• Pages: 1-26
• DOI: https://doi.org/10.15346/hc.v7i1
• Public URL: https://nottingham-repository.worktribe.com/output/4905179
• Publisher URL: https://hcjournal.org/index.php/jhc/article/view/111