An external validation of the QCovid risk prediction algorithm for risk of mortality from COVID-19 in adults: a national validation cohort study in England

Nafilyan, Vahé; Humberstone, Ben; Mehta, Nisha; Diamond, Ian; Coupland, Carol; Lorenzi, Luke; Pawelek, Piotr; Schofield, Ryan; Morgan, Jasper; Brown, Paul; Lyons, Ronan; Sheikh, Aziz; Hippisley-Cox, Julia

Abstract

Background: Public policy measures and clinical risk assessments relevant to COVID-19 need to be aided by risk prediction models that are rigorously developed and validated. We aimed to externally validate a risk prediction algorithm (QCovid) to estimate mortality outcomes from COVID-19 in adults in England. Methods: We did a population-based cohort study using the UK Office for National Statistics Public Health Linked Data Asset, a cohort of individuals aged 19–100 years, based on the 2011 census and linked to Hospital Episode Statistics, the General Practice Extraction Service data for pandemic planning and research, and radiotherapy and systemic chemotherapy records. The primary outcome was time to COVID-19 death, defined as confirmed or suspected COVID-19 death as per death certification. Two periods were used: (1) Jan 24 to April 30, 2020, and (2) May 1 to July 28, 2020. We assessed the performance of the QCovid algorithms using measures of discrimination and calibration. Using predicted 90-day risk of COVID-19 death, we calculated r2 values, Brier scores, and measures of discrimination and calibration with corresponding 95% CIs over the two time periods. Findings: We included 34 897 648 adults aged 19–100 years resident in England. 26 985 (0·08%) COVID-19 deaths occurred during the first period and 13 177 (0·04%) during the second. The algorithms had good discrimination and calibration in both periods. In the first period, they explained 77·1% (95% CI 76·9–77·4) of the variation in time to death in men and 76·3% (76·0–76·6) in women. The D statistic was 3·761 (3·732–3·789) for men and 3·671 (3·640–3·702) for women and Harrell's C was 0·935 (0·933–0·937) for men and 0·945 (0·943–0·947) for women. Similar results were obtained for the second time period. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths in the first period was 65·94% for men and 71·67% for women. Interpretation: The QCovid population-based risk algorithm performed well, showing high levels of discrimination for COVID-19 deaths in men and women for both time periods. QCovid has the potential to be dynamically updated as the pandemic evolves and, therefore, has potential use in guiding national policy. Funding: UK National Institute for Health Research.

Citation

Nafilyan, V., Humberstone, B., Mehta, N., Diamond, I., Coupland, C., Lorenzi, L., Pawelek, P., Schofield, R., Morgan, J., Brown, P., Lyons, R., Sheikh, A., & Hippisley-Cox, J. (2021). An external validation of the QCovid risk prediction algorithm for risk of mortality from COVID-19 in adults: a national validation cohort study in England. The Lancet. Digital Health, 3(7), E425-E433. https://doi.org/10.1016/S2589-7500%2821%2900080-7

• Journal Article Type: Article
• Acceptance Date: Apr 23, 2021
• Online Publication Date: Jun 25, 2021
• Publication Date: Jul 1, 2021
• Deposit Date: Jul 8, 2021
• Publicly Available Date: Jul 14, 2021
• Journal: The Lancet Digital Health
• Print ISSN: 2589-7500
• Electronic ISSN: 2589-7500
• Peer Reviewed: Peer Reviewed
• Volume: 3
• Issue: 7
• Pages: E425-E433
• DOI: https://doi.org/10.1016/S2589-7500%2821%2900080-7
• Public URL: https://nottingham-repository.worktribe.com/output/5621414
• Publisher URL: https://www.thelancet.com/journals/landig/article/PIIS2589-7500(21)00080-7/fulltext
• Related Public URLs: https://www.sciencedirect.com/science/article/pii/S2589750021000807