Improving Uncertainty Estimation With Semi-supervised Deep Learning for COVID-19 Detection Using Chest X-ray Images

Calderon-Ramirez, Saul; Yang, Shengxiang; Moemeni, Armaghan; Colreavy-Donnelly, Simon; Elizondo, David A.; Oala, Luis; Rodríguez-Capitán, Jorge; Jiménez-Navarro, Manuel; López-Rubio, Ezequiel; Molina-Cabello, Miguel A.

Abstract

In this work we implement a COVID-19 infection detection system based on chest Xray images with uncertainty estimation. Uncertainty estimation is vital for safe usage of computer aided diagnosis tools in medical applications. Model estimations with high uncertainty should be carefully analyzed by a trained radiologist. We aim to improve uncertainty estimations using unlabelled data through the MixMatch semi-supervised framework. We test popular uncertainty estimation approaches, comprising Softmax scores, Monte-Carlo dropout and deterministic uncertainty quantification. To compare the reliability of the uncertainty estimates, we propose the usage of the Jensen-Shannon distance between the uncertainty distributions of correct and incorrect estimations. This metric is statistically relevant, unlike most previously used metrics, which often ignore the distribution of the uncertainty estimations. Our test results show a significant improvement in uncertainty estimates when using unlabelled data. The best results are obtained with the use of the Monte Carlo dropout method.

Citation

Calderon-Ramirez, S., Yang, S., Moemeni, A., Colreavy-Donnelly, S., Elizondo, D. A., Oala, L., Rodríguez-Capitán, J., Jiménez-Navarro, M., López-Rubio, E., & Molina-Cabello, M. A. (2021). Improving Uncertainty Estimation With Semi-supervised Deep Learning for COVID-19 Detection Using Chest X-ray Images. IEEE Access, 9, 85442 - 85454. https://doi.org/10.1109/ACCESS.2021.3085418

• Journal Article Type: Article
• Acceptance Date: May 22, 2021
• Online Publication Date: Jun 2, 2021
• Publication Date: 2021
• Deposit Date: Jun 4, 2021
• Publicly Available Date: Jun 8, 2021
• Journal: IEEE Access
• Electronic ISSN: 2169-3536
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 9
• Pages: 85442 - 85454
• DOI: https://doi.org/10.1109/ACCESS.2021.3085418
• Keywords: Uncertainty estimation , Coronavirus , Covid-19 , Chest X-Ray , Computer Aided Diagnosis , Semi-Supervised Deep Learning , MixMatch
• Public URL: https://nottingham-repository.worktribe.com/output/5625937
• Publisher URL: https://ieeexplore.ieee.org/document/9445026