Real-world Independent Testing of e-ASPECTS Software (RITeS): statistical analysis plan

Mair, Grant; Chappell, Francesca; Martin, Chloe; Dye, David; Bath, Philip M.; Muir, Keith W.; von Kummer, R�diger; Al-Shahi Salman, Rustam; Sandercock, Peter A. G.; Macleod, Malcolm; Sprigg, Nikola; White, Philip; Wardlaw, Joanna M.

Abstract

Background: Artificial intelligence-based software may automatically detect ischaemic stroke lesions and provide an Alberta Stroke Program Early CT score (ASPECTS) on CT, and identify arterial occlusion and provide a collateral score on CTA. Large-scale independent testing will inform clinical use, but is lacking. We aim to test e-ASPECTS and e-CTA (Brainomix, Oxford UK) using CT scans obtained from a range of clinical studies.
Methods: Using prospectively collected baseline CT and CTA scans from 10 national/international clinical stroke trials or registries (total >6600 patients), we will select a large clinically representative sample for testing e-ASPECTS and e-CTA compared to previously acquired independent expert human interpretation (reference standard). Our primary aims are to test agreement between software-derived and masked human expert ASPECTS, and the diagnostic accuracy of e-ASPECTS for identifying all causes of stroke symptoms using follow-up imaging and final clinical opinion as diagnostic ground truth. Our secondary aims are to test when and why e-ASPECTS is more or less accurate, or succeeds/fails to produce results, agreement between e-CTA and human expert CTA interpretation, and repeatability of e-ASPECTS/e-CTA results. All testing will be conducted on an intention-to-analyse basis. We will assess agreement between software and expert-human ratings and test the diagnostic accuracy of software.
Conclusions: RITeS will provide comprehensive, robust and representative testing of e-ASPECTS and e-CTA against the current gold-standard, expert-human interpretation.

Citation

Mair, G., Chappell, F., Martin, C., Dye, D., Bath, P. M., Muir, K. W., …Wardlaw, J. M. (2020). Real-world Independent Testing of e-ASPECTS Software (RITeS): statistical analysis plan. AMRC Open Research, 2, Article 20. https://doi.org/10.12688/amrcopenres.12904.1

• Journal Article Type: Article
• Acceptance Date: Apr 10, 2020
• Online Publication Date: Apr 28, 2020
• Publication Date: Apr 28, 2020
• Deposit Date: Apr 30, 2020
• Publicly Available Date: Apr 30, 2020
• Journal: AMRC Open Research
• Publisher: Association of Medical Research Charities
• Peer Reviewed: Not Peer Reviewed
• Volume: 2
• Article Number: 20
• DOI: https://doi.org/10.12688/amrcopenres.12904.1
• Public URL: https://nottingham-repository.worktribe.com/output/4365184
• Publisher URL: https://amrcopenresearch.org/articles/2-20/v1
• Additional Information: Referee status: Awaiting Peer Review; Grant Information: We are grateful to the Stroke Association for commissioning and principally funding the RITeS study (TSA CR 2017/01).; ; We also acknowledge the MRC (Medical Research Council) Proximity to Discovery fund for supporting our purchase of an e-ASPECTS software licence. ; ; GM is in receipt of the Stroke Association Edith Murphy Foundation Senior Clinical Lectureship for Medical Professionals (SA L-SMP 18/1000) and the 2018 Royal College of Radiologists Pump-Priming grant. Both projects include aims to develop deep learning methods for the automated detection of stroke lesions on CT.; PMB is Stroke Association Professor of Stroke Medicine and is a NIHR Senior Investigator.; RASS reports grants from the British Heart Foundation, The Stroke Association, and the Medical Research Council. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.; Copyright: This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.