Human judgement forecasting tournaments: A feasibility study based on the COVID-19 pandemic with public health practitioners in England

Davies, Nathan; Ferris, Simon

Abstract

Objectives: To explore the viability of running human judgement forecasting tournaments with public health practitioners, and to gather initial data on forecasting accuracy and participant perceptions of forecasting. Study design: Quality improvement study comprising two COVID-19 forecasting tournaments using Brier Skill Score scoring and a follow-up participant questionnaire. Methods: Over two forecasting tournaments, public health registrars in the East Midlands, UK, assigned probabilities to future possible binary events relating to COVID-19. Participants also completed a questionnaire on their experiences of forecasting. Results: There were 17 participants in the first tournament and nine in the second tournament, with no new participants. In both tournaments, the majority of participants scored a Brier Skill Score above the benchmark of 0. The median Brier Skill Score improved slightly between the two tournaments. Participants reported luck and changing political climates as impacting their performance. Participants reported forecasting in their day job but had received no formal training to do so. Conclusions: Forecasting is an important public health skill, and human judgement forecasting tournaments can be run amongst public health practitioners with little time and resource requirements. Further research would help identify whether training, teamwork or other interventions can improve public health forecasting accuracy.

Citation

Davies, N., & Ferris, S. (2022). Human judgement forecasting tournaments: A feasibility study based on the COVID-19 pandemic with public health practitioners in England. Public Health In Practice, 3, Article 100260. https://doi.org/10.1016/j.puhip.2022.100260

• Journal Article Type: Article
• Acceptance Date: Apr 16, 2022
• Online Publication Date: Apr 22, 2022
• Publication Date: Jun 1, 2022
• Deposit Date: May 17, 2022
• Publicly Available Date: May 18, 2022
• Journal: Public Health in Practice
• Electronic ISSN: 2666-5352
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 3
• Article Number: 100260
• DOI: https://doi.org/10.1016/j.puhip.2022.100260
• Keywords: Public Health, Environmental and Occupational Health; Health Policy
• Public URL: https://nottingham-repository.worktribe.com/output/8130711
• Publisher URL: https://www.sciencedirect.com/science/article/pii/S2666535222000362?via%3Dihub