Relationship Between Mammography Readers Real-life Performance and Performance in a Test set Based Assessment Scheme in a National Breast Screening Programme

Chen, Yan; James, Jonathan J.; Cornford, Eleanor J.; Jenkins, Jacquie

Abstract

Purpose

To compare an individual’s Personal Performance in Mammographic Screening (PERFORMS) score with their Breast Screening Information System (BSIS) real-life performance data and determine which parameters in the PERFORMS scheme offer the best reflection of BSIS real-life performance metrics.

Materials and Methods

In this retrospective study, the BSIS real-life performance metrics of individual readers (n = 452) in the National Health Service Breast Screening Program (NHSBSP) in England were compared with performance in the test set–based assessment scheme over a 3-year period from 2013 to 2016. Cancer detection rate (CDR), recall rate, and positive predictive value (PPV) were calculated for each reader, for both real-life screening and the PERFORMS test. For each metric, real-life and test set versions were compared using a Pearson correlation. The real-life CDR, recall rate, and PPV of outliers were compared against other readers (nonoutliers) using analysis of variance.

Results

BSIS real-life CDRs, recall rates, and PPVs showed positive correlations with the equivalent PERFORMS measures (P < .001, P = .002, and P < .001, respectively). The mean real-life CDR of PERFORMS outliers was 7.2 per 1000 women screened and was significantly lower than other readers (nonoutliers) where the real-life CDR was 7.9 (P = .002). The mean real-life screening PPV of PERFORMS outliers was 0.14% and was significantly lower than the nonoutlier group who had a mean PPV of 0.17% (P = .006).

Conclusion

The use of test set–based assessment schemes in a breast screening program has the potential to predict and identify poor performance in real life.

Citation

Chen, Y., James, J. J., Cornford, E. J., & Jenkins, J. (2020). Relationship Between Mammography Readers Real-life Performance and Performance in a Test set Based Assessment Scheme in a National Breast Screening Programme. Radiology, 2(5), https://doi.org/10.1148/rycan.2020200016

• Journal Article Type: Article
• Acceptance Date: May 2, 2020
• Online Publication Date: Sep 25, 2020
• Publication Date: Sep 25, 2020
• Deposit Date: Jun 10, 2020
• Publicly Available Date: Mar 26, 2021
• Journal: Radiology
• Print ISSN: 0033-8419
• Publisher: Radiological Society of North America
• Peer Reviewed: Peer Reviewed
• Volume: 2
• Issue: 5
• DOI: https://doi.org/10.1148/rycan.2020200016
• Public URL: https://nottingham-repository.worktribe.com/output/4617698
• Publisher URL: https://pubs.rsna.org/doi/full/10.1148/rycan.2020200016