SEGAL time series classification — Stable explanations using a generative model and an adaptive weighting method for LIME

Meng, Han; Wagner, Christian; Triguero, Isaac

Abstract

Local Interpretability Model-agnostic Explanations (LIME) is a well-known post-hoc technique for explaining black-box models. While very useful, recent research highlights challenges around the explanations generated. In particular, there is a potential lack of stability, where the explanations provided vary over repeated runs of the algorithm, casting doubt on their reliability. This paper investigates the stability of LIME when applied to multivariate time series classification. We demonstrate that the traditional methods for generating neighbours used in LIME carry a high risk of creating ‘fake’ neighbours, which are out-of-distribution in respect to the trained model and far away from the input to be explained. This risk is particularly pronounced for time series data because of their substantial temporal dependencies. We discuss how these out-of-distribution neighbours contribute to unstable explanations. Furthermore, LIME weights neighbours based on user-defined hyperparameters which are problem-dependent and hard to tune. We show how unsuitable hyperparameters can impact the stability of explanations. We propose a two-fold approach to address these issues. First, a generative model is employed to approximate the distribution of the training data set, from which within-distribution samples and thus meaningful neighbours can be created for LIME. Second, an adaptive weighting method is designed in which the hyperparameters are easier to tune than those of the traditional method. Experiments on real-world data sets demonstrate the effectiveness of the proposed method in providing more stable explanations using the LIME framework. In addition, in-depth discussions are provided on the reasons behind these results.

Citation

Meng, H., Wagner, C., & Triguero, I. (2024). SEGAL time series classification — Stable explanations using a generative model and an adaptive weighting method for LIME. Neural Networks, 176, 106345. https://doi.org/10.1016/j.neunet.2024.106345

• Journal Article Type: Article
• Acceptance Date: Apr 25, 2024
• Online Publication Date: Apr 27, 2024
• Publication Date: Aug 1, 2024
• Deposit Date: Mar 6, 2025
• Publicly Available Date: Mar 11, 2025
• Journal: Neural Networks
• Print ISSN: 0893-6080
• Electronic ISSN: 1879-2782
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 176
• Pages: 106345
• DOI: https://doi.org/10.1016/j.neunet.2024.106345
• Public URL: https://nottingham-repository.worktribe.com/output/34350801
• Publisher URL: https://www.sciencedirect.com/science/article/pii/S0893608024002697?via%3Dihub
• Additional Information: This article is maintained by: Elsevier; Article Title: SEGAL time series classification — Stable explanations using a generative model and an adaptive weighting method for LIME; Journal Title: Neural Networks; CrossRef DOI link to publisher maintained version: https://doi.org/10.1016/j.neunet.2024.106345; Content Type: article; Copyright: © 2024 The Author(s). Published by Elsevier Ltd.