Measuring fidelity to manualised peer support for people with severe mental health conditions: development and psychometric evaluation of the UPSIDES Fidelity Scale

Hiltensperger, Ramona; Kotera, Yasuhiro; Wolf, Philip; Nixdorf, Rebecca; Charles, Ashleigh; Farkas, Marianne; Grayzman, Alina; Kalha, Jasmine; Korde, Palak; Mahlke, Candelaria; Moran, Galia; Mpango, Richard; Mtei, Rachel; Ryan, Grace; Shamba, Donat; Wenzel, Lisa; Slade, Mike; Puschner, Bernd

Abstract

Background. Peer support workers provide support for people experiencing mental health conditions based on their own lived experience of mental health problems. Assessing fidelity to core ingredients of peer support is vital for successful implementation and intervention delivery. Modifications to its implementation are needed when scaling up to different socio-economic settings, raising further uncertainty about fidelity. As part of a large multi-centre study on peer support called Using Peer Support In Developing Empowering Mental Health Services (UPSIDES), we developed and evaluated the psychometric properties of the UPSIDES Fidelity Scale.

Methods. We constructed the fidelity scale based on an initial item pool developed through international expert consultation and iterative feedback. Scale refinement involved site-level expert consultation and translation, resulting in a service userrated 28-item version and a peer support worker-rated 21-item version assessing receipt, engagement, enactment, competence, communication and peer supportspecific components. Both versions are available in six languages: English, German, Luganda, Kiswahili, Hebrew and Gujarati. The scale was then evaluated at six study sites across five countries, with peer support workers and their clients completing their respective ratings four and eight months after initial peer support worker contact. Psychometric evaluation included analysis of internal consistency, construct validity and criterion validity.

Results. For the 315 participants, item statistics showed a skewed distribution of fidelity values but no restriction of range. Internal consistency was adequate (range α = 0.675 to 0.969) for total scores and all subscales in both versions. Confirmatory 4 factor analysis indicated acceptable fit of the proposed factor structure for the service user version (χ2/df = 2.746; RMSEA = 0.084) and moderate fit for the peer support worker version (χ2/df = 3.087; RMSEA = 0.093). Both versions showed significant correlations with external criteria: number of peer support sessions; perceived recovery orientation of the intervention; and severity of illness.

Conclusions. The scale demonstrates good reliability, construct and criterion validity, making it a pragmatic and psychometrically acceptable measure for assessing fidelity to a manualised peer support worker intervention. Recommendations for use, along with research and practical implications, are addressed. As validated, multi-lingual tool that adapts to diverse settings this scale is uniquely positioned for global application.

Citation

Hiltensperger, R., Kotera, Y., Wolf, P., Nixdorf, R., Charles, A., Farkas, M., Grayzman, A., Kalha, J., Korde, P., Mahlke, C., Moran, G., Mpango, R., Mtei, R., Ryan, G., Shamba, D., Wenzel, L., Slade, M., & Puschner, B. (2024). Measuring fidelity to manualised peer support for people with severe mental health conditions: development and psychometric evaluation of the UPSIDES Fidelity Scale. BMC Psychiatry, 24, Article 675. https://doi.org/10.1186/s12888-024-06081-8

• Journal Article Type: Article
• Acceptance Date: Sep 12, 2024
• Online Publication Date: Oct 11, 2024
• Publication Date: Oct 11, 2024
• Deposit Date: Sep 16, 2024
• Publicly Available Date: Sep 16, 2024
• Journal: BMC Psychiatry
• Electronic ISSN: 1471-244X
• Publisher: Springer Verlag
• Peer Reviewed: Peer Reviewed
• Volume: 24
• Article Number: 675
• DOI: https://doi.org/10.1186/s12888-024-06081-8
• Public URL: https://nottingham-repository.worktribe.com/output/39715529
• Publisher URL: https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-024-06081-8