Using GAMMs to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit

Thul, R�diger; Conklin, Kathy; Barr, Dale J

Abstract

Data from each subject in a repeated-measures experiment forms a time series , which may include trial-by-trial fluctuations arising from human factors such as practice or fatigue. Concerns about the statistical implications of such effects have increased the popularity of Generalized Additive Mixed Models (GAMMs), a powerful technique for modeling wiggly patterns. We question these statistical concerns and investigate the costs and benefits of using GAMMs relative to linear mixed-effects models (LMEMs). In a Monte Carlo simulation study, LMEMs that ignored time-varying effects were no more prone to false positives than GAMMs. Although GAMMs generally boosted power for within-subject effects, they reduced power for between-subject effects, sometimes to a severe degree. Our results signal the importance of proper subject-level randomization as the main defense against statistical artifacts due to by-trial fluctuations.

Citation

Thul, R., Conklin, K., & Barr, D. J. (2021). Using GAMMs to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit. Journal of Memory and Language, 120, Article 104247. https://doi.org/10.1016/j.jml.2021.104247

• Journal Article Type: Article
• Acceptance Date: Mar 31, 2021
• Online Publication Date: Apr 20, 2021
• Publication Date: Oct 1, 2021
• Deposit Date: Apr 14, 2021
• Publicly Available Date: Apr 21, 2022
• Journal: Journal of Memory and Language
• Print ISSN: 0749-596X
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 120
• Article Number: 104247
• DOI: https://doi.org/10.1016/j.jml.2021.104247
• Public URL: https://nottingham-repository.worktribe.com/output/5466193
• Publisher URL: https://www.sciencedirect.com/science/article/pii/S0749596X21000309