Skip to main content

Research Repository

Advanced Search

Funnel plots may show asymmetry in the absence of publication bias with continuous outcomes dependent on baseline risk: presentation of a new publication bias test

Doleman, Brett; Freeman, Suzanne C.; Lund, Jonathan N.; Williams, John P.; Sutton, Alex J.

Authors

Brett Doleman

Suzanne C. Freeman

JOHN WILLIAMS john.williams7@nottingham.ac.uk
Clinical Associate Professor

Alex J. Sutton



Abstract

This study aimed to determine for continuous outcomes dependent on baseline risk, whether funnel plot asymmetry may be due to statistical artefact rather than publication bias and evaluate a novel test to resolve this. Firstly, we conducted assessment for publication bias in nine meta‐analyses of postoperative analgesics (344 trials with 25 348 participants). Secondly, we attempted to resolve the observed asymmetry by considering meta‐regression residuals as outcome (rather than mean difference) and (inverse) sample size as the exploratory variable (rather than SE). Since the approach resolved the asymmetry, we evaluated it, and related approaches, using a simulation study considering four scenarios comprised of every combination of baseline interactions and absolute selective publication bias being present or not (10 000 simulated meta‐analyses per scenario with no residual between‐study heterogeneity). The test based on meta‐regression residuals and inverse sample size performed as well as conventional tests (Egger's test) when no baseline risk was present and reduced type I errors when baseline risk was present. It also had modest power to detect publication bias in the presence of baseline risk. We demonstrated that correlation between effect estimates and SEs produces funnel plot asymmetry in the presence of no publication bias for continuous outcomes dependent on baseline risk. Our novel approach of assessing funnel plot asymmetry using a modified funnel plot and test based on residuals and inverse sample size may have improved performance when carrying out publication bias assessments for unstandardized mean differences where treatment effects are dependent on baseline risk.

Journal Article Type Article
Publication Date 2020-07
Journal Research Synthesis Methods
Print ISSN 1759-2879
Electronic ISSN 1759-2887
Publisher Wiley
Peer Reviewed Peer Reviewed
Volume 11
Issue 4
Pages 522-534
APA6 Citation Doleman, B., Freeman, S. C., Lund, J. N., Williams, J. P., & Sutton, A. J. (2020). Funnel plots may show asymmetry in the absence of publication bias with continuous outcomes dependent on baseline risk: presentation of a new publication bias test. Research Synthesis Methods, 11(4), 522-534. https://doi.org/10.1002/jrsm.1414
DOI https://doi.org/10.1002/jrsm.1414
Publisher URL https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1414
;