Accuracy of artificial intelligence software for CT angiography in stroke

Mair, Grant; White, Philip; Bath, Philip M.; Muir, Keith; Martin, Chloe; Dye, David; Chappell, Francesca; von Kummer, Rüdiger; Macleod, Malcolm; Sprigg, Nikola; Wardlaw, Joanna M.; for the RITeS Collaboration

Abstract

Objective: Software developed using artificial intelligence may automatically identify arterial occlusion and provide collateral vessel scoring on CT angiography (CTA) performed acutely for ischemic stroke. We aimed to assess the diagnostic accuracy of e‐CTA by Brainomix™ Ltd by large‐scale independent testing using expert reading as the reference standard. Methods: We identified a large clinically representative sample of baseline CTA from 6 studies that recruited patients with acute stroke symptoms involving any arterial territory. We compared e‐CTA results with masked expert interpretation of the same scans for the presence and location of laterality‐matched arterial occlusion and/or abnormal collateral score combined into a single measure of arterial abnormality. We tested the diagnostic accuracy of e‐CTA for identifying any arterial abnormality (and in a sensitivity analysis compliant with the manufacturer's guidance that software only be used to assess the anterior circulation). Results: We include CTA from 668 patients (50% female; median: age 71 years, NIHSS 9, 2.3 h from stroke onset). Experts identified arterial occlusion in 365 patients (55%); most (343, 94%) involved the anterior circulation. Software successfully processed 545/668 (82%) CTAs. The sensitivity, specificity and diagnostic accuracy of e‐CTA for detecting arterial abnormality were each 72% (95% CI = 66–77%). Diagnostic accuracy was non‐significantly improved in a sensitivity analysis excluding occlusions from outside the anterior circulation (76%, 95% CI = 72–80%). Interpretation: Compared to experts, the diagnostic accuracy of e‐CTA for identifying acute arterial abnormality was 72–76%. Users of e‐CTA should be competent in CTA interpretation to ensure all potential thrombectomy candidates are identified.

• Journal Article Type: Article
• Acceptance Date: May 1, 2023
• Online Publication Date: May 19, 2023
• Publication Date: 2023-07
• Deposit Date: May 22, 2023
• Publicly Available Date: May 22, 2023
• Journal: Annals of Clinical and Translational Neurology
• Electronic ISSN: 2328-9503
• Publisher: Wiley
• Peer Reviewed: Peer Reviewed
• Volume: 10
• Issue: 7
• Pages: 1072-1082
• DOI: https://doi.org/10.1002/acn3.51790
• Keywords: Neurology (clinical); General Neuroscience
• Public URL: https://nottingham-repository.worktribe.com/output/21092990
• Publisher URL: https://onlinelibrary.wiley.com/doi/full/10.1002/acn3.51790