Tools and methods for evaluating the change to health service delivery due to pandemics or other similar emergencies: A rapid evidence review.

Keegan, Dylan; Heffernan, Eithne; Clarke, Bridget; Deasy, Conor; O'Donnell, Cathal; Crowley, Philip; Hughes, Angela; Murphy, Andrew W; Masterson, Siobhán

Abstract

The novel coronavirus, SARS-CoV-2 and its associated disease COVID-19, were declared a pandemic in March 2020. Countries developed rapid response activities within their health services to prevent spread of the virus and protect their populations. Evaluating health service delivery change is vital to assess how adapted practices worked, particularly during times of crisis. This review examined tools and methods that are used to evaluate health service delivery change during pandemics and similar emergencies. Five databases were searched, including PubMed, CENTRAL, Embase, CINAHL, and PsycINFO. The SPIDER tool informed the inclusion criteria for the articles. Articles in English and published from 2002 to 2020 were included. Risk of bias was assessed using the Mixed-Methods Appraisal Tool (MMAT). A narrative synthesis approach was used to analyse the studies. Eleven articles met the inclusion criteria. Many evaluation tools, methods, and frameworks were identified in the literature. Only one established tool was specific to a particular disease outbreak. Others, including rapid-cycle improvement and PDSA cycles were implemented across various disease outbreaks. Novel evaluation strategies were common across the literature and included checklists, QI frameworks, questionnaires, and surveys. Adherence practices, experience with telehealth, patient/healthcare staff safety, and clinical competencies were some areas evaluated by the tools and methods. Several domains, including patient/practitioner safety and patient/practitioner experience with telemedicine were also identified in the studies.

Citation

Keegan, D., Heffernan, E., Clarke, B., Deasy, C., O'Donnell, C., Crowley, P., Hughes, A., Murphy, A. W., & Masterson, S. (2024). Tools and methods for evaluating the change to health service delivery due to pandemics or other similar emergencies: A rapid evidence review. Evaluation and Program Planning, 102, Article 102378. https://doi.org/10.1016/j.evalprogplan.2023.102378

• Journal Article Type: Article
• Acceptance Date: Sep 20, 2023
• Online Publication Date: Sep 22, 2023
• Publication Date: 2024-02
• Deposit Date: Dec 18, 2023
• Publicly Available Date: Sep 23, 2025
• Journal: Evaluation and Program Planning
• Print ISSN: 0149-7189
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 102
• Article Number: 102378
• DOI: https://doi.org/10.1016/j.evalprogplan.2023.102378
• Keywords: Imposed delivery change, Change implementation, Quality improvement, Pandemic, Evaluation strategies, Public health
• Public URL: https://nottingham-repository.worktribe.com/output/27069844
• Publisher URL: https://www.sciencedirect.com/science/article/pii/S0149718923001556?via%3Dihub