Funnel plots may show asymmetry in the absence of publication bias with continuous outcomes dependent on baseline risk: presentation of a new publication bias test

Doleman, Brett; Freeman, Suzanne C.; Lund, Jonathan N.; Williams, John P.; Sutton, Alex J.

Abstract

This study aimed to determine for continuous outcomes dependent on baseline risk, whether funnel plot asymmetry may be due to statistical artefact rather than publication bias and evaluate a novel test to resolve this. Firstly, we conducted assessment for publication bias in nine meta‐analyses of postoperative analgesics (344 trials with 25 348 participants). Secondly, we attempted to resolve the observed asymmetry by considering meta‐regression residuals as outcome (rather than mean difference) and (inverse) sample size as the exploratory variable (rather than SE). Since the approach resolved the asymmetry, we evaluated it, and related approaches, using a simulation study considering four scenarios comprised of every combination of baseline interactions and absolute selective publication bias being present or not (10 000 simulated meta‐analyses per scenario with no residual between‐study heterogeneity). The test based on meta‐regression residuals and inverse sample size performed as well as conventional tests (Egger's test) when no baseline risk was present and reduced type I errors when baseline risk was present. It also had modest power to detect publication bias in the presence of baseline risk. We demonstrated that correlation between effect estimates and SEs produces funnel plot asymmetry in the presence of no publication bias for continuous outcomes dependent on baseline risk. Our novel approach of assessing funnel plot asymmetry using a modified funnel plot and test based on residuals and inverse sample size may have improved performance when carrying out publication bias assessments for unstandardized mean differences where treatment effects are dependent on baseline risk.

Citation

Doleman, B., Freeman, S. C., Lund, J. N., Williams, J. P., & Sutton, A. J. (2020). Funnel plots may show asymmetry in the absence of publication bias with continuous outcomes dependent on baseline risk: presentation of a new publication bias test. Research Synthesis Methods, 11(4), 522-534. https://doi.org/10.1002/jrsm.1414

• Journal Article Type: Article
• Acceptance Date: Apr 23, 2020
• Online Publication Date: May 6, 2020
• Publication Date: 2020-07
• Deposit Date: Jun 29, 2020
• Publicly Available Date: Jun 29, 2020
• Journal: Research Synthesis Methods
• Print ISSN: 1759-2879
• Electronic ISSN: 1759-2887
• Publisher: Wiley
• Peer Reviewed: Peer Reviewed
• Volume: 11
• Issue: 4
• Pages: 522-534
• DOI: https://doi.org/10.1002/jrsm.1414
• Public URL: https://nottingham-repository.worktribe.com/output/4482702
• Publisher URL: https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1414