Variable selection for inferential models with relatively high-dimensional data: Between method heterogeneity and covariate stability as adjuncts to robust selection

Lima, Eliana; Davies, Peers; Kaler, Jasmeet; Lovatt, Fiona; Green, Martin

Abstract

Variable selection in inferential modelling is problematic when the number of variables is large relative to the number of data points, especially when multicollinearity is present. A variety of techniques have been described to identify ‘important’ subsets of variables from within a large parameter space but these may produce different results which creates difficulties with inference and reproducibility. Our aim was evaluate the extent to which variable selection would change depending on statistical approach and whether triangulation across methods could enhance data interpretation. A real dataset containing 408 subjects, 337 explanatory variables and a normally distributed outcome was used. We show that with model hyperparameters optimised to minimise cross validation error, ten methods of automated variable selection produced markedly different results; different variables were selected and model sparsity varied greatly. Comparison between multiple methods provided valuable additional insights. Two variables that were consistently selected and stable across all methods accounted for the majority of the explainable variability; these were the most plausible important candidate variables. Further variables of importance were identified from evaluating selection stability across all methods. In conclusion, triangulation of results across methods, including use of covariate stability, can greatly enhance data interpretation and confidence in variable selection.

Citation

Lima, E., Davies, P., Kaler, J., Lovatt, F., & Green, M. (2020). Variable selection for inferential models with relatively high-dimensional data: Between method heterogeneity and covariate stability as adjuncts to robust selection. Scientific Reports, 10, https://doi.org/10.1038/s41598-020-64829-0

• Journal Article Type: Article
• Acceptance Date: Apr 17, 2020
• Online Publication Date: May 14, 2020
• Publication Date: May 14, 2020
• Deposit Date: May 15, 2020
• Publicly Available Date: May 15, 2020
• Journal: Scientific Reports
• Print ISSN: 2045-2322
• Electronic ISSN: 2045-2322
• Publisher: Nature Publishing Group
• Peer Reviewed: Peer Reviewed
• Volume: 10
• Article Number: 8002
• DOI: https://doi.org/10.1038/s41598-020-64829-0
• Keywords: Multidisciplinary
• Public URL: https://nottingham-repository.worktribe.com/output/4440810
• Publisher URL: https://www.nature.com/articles/s41598-020-64829-0
• Additional Information: Received: 29 January 2020; Accepted: 17 April 2020; First Online: 14 May 2020; : The authors declare no competing interests.