Self-Adaptive Particle Swarm Optimization-Based Echo State Network for Time Series Prediction

Xue, Yu; Zhang, Qi; Neri, Ferrante

Abstract

Echo state networks (ESNs), belonging to the family of recurrent neural networks (RNNs), are suitable for addressing complex nonlinear tasks due to their rich dynamic characteristics and easy implementation. The reservoir of the ESN is composed of a large number of sparsely connected neurons with randomly generated weight matrices. How to set the structural parameters of the ESN becomes a difficult problem in practical applications. Traditionally, the design of the parameters of the ESN structure is performed manually. The manual adjustment of the ESN parameters is not convenient since it is an extremely challenging and time-consuming task. This paper proposes an ensemble of five particle swarm optimization (PSO) strategies to design the structure of ESN and then reduce the manual intervention in the design process. An adaptive selection mechanism is used for each particle in the evolution to select a strategy from the strategy candidate pool for evolution. In addition, leaky integration neurons are used as reservoir internal neurons, which are added within the adaptive mechanism for optimization. The root mean squared error (RMSE) is adopted as the evaluation criterion. The experimental results on Mackey-Glass time series benchmark dataset show that the proposed method outperforms other traditional evolutionary methods. Furthermore, experimental results on electrocardiogram dataset show that the proposed method on the ensemble of PSO displays an excellent performance on real-world problems.

Citation

Xue, Y., Zhang, Q., & Neri, F. (2021). Self-Adaptive Particle Swarm Optimization-Based Echo State Network for Time Series Prediction. International Journal of Neural Systems, 31(12), Article 2150057. https://doi.org/10.1142/s012906572150057x

• Journal Article Type: Article
• Acceptance Date: Sep 14, 2021
• Online Publication Date: Oct 28, 2021
• Publication Date: 2021-12
• Deposit Date: Sep 14, 2021
• Publicly Available Date: Oct 29, 2022
• Journal: International Journal of Neural Systems
• Print ISSN: 0129-0657
• Electronic ISSN: 1793-6462
• Publisher: World Scientific Pub Co Pte Ltd
• Peer Reviewed: Peer Reviewed
• Volume: 31
• Issue: 12
• Article Number: 2150057
• DOI: https://doi.org/10.1142/s012906572150057x
• Keywords: Computer Networks and Communications; General Medicine
• Public URL: https://nottingham-repository.worktribe.com/output/6238890
• Publisher URL: https://www.worldscientific.com/doi/10.1142/S012906572150057X
• Additional Information: Electronic version of an article published as International Journal of Neural Systems, Vol. 31, no. 12, 2021, https://doi.org/10.1142/S012906572150057X © copyright World Scientific Publishing Company https://www.worldscientific.com/worldscinet/ijns