Advanced multimodal fusion method for very short-term solar irradiance forecasting using sky images and meteorological data: A gate and transformer mechanism approach

Zhang, Liwenbo; Wilson, Robin; Sumner, Mark; Wu, Yupeng

Abstract

Cloud dynamics are the main factor influencing the intermittent variability of short-term solar irradiance, and therefore affect the solar farm output. Sky images have been widely used for short-term solar irradiance prediction with encouraging results due to the spatial information they contain. At present, there is little discussion on the most promising deep learning methods to integrate images with quantitative measures of solar irradiation. To address this gap, we optimise the current mainstream framework using gate architecture and propose a new transformer-based framework in an attempt to achieve better prediction results. It was found that compared to the classical CNN model based on late feature-level fusion, the transformer framework model based on early feature-level prediction improves the balanced accuracy of ramp events by 9.43% and 3.91% on the 2-min and 6-min scales, respectively. However, based on the results, it can be concluded that for the single picture-digital bimodal model, the spatial information validity of a single picture is difficult to achieve beyond 10 min. This work has the potential to contribute to the interpretability and iterability of deep learning models based on sky images.

Citation

Zhang, L., Wilson, R., Sumner, M., & Wu, Y. (2023). Advanced multimodal fusion method for very short-term solar irradiance forecasting using sky images and meteorological data: A gate and transformer mechanism approach. Renewable Energy, 216, Article 118952. https://doi.org/10.1016/j.renene.2023.118952

• Journal Article Type: Article
• Acceptance Date: Jun 22, 2023
• Online Publication Date: Jun 28, 2023
• Publication Date: 2023-11
• Deposit Date: Jun 29, 2023
• Publicly Available Date: Jun 29, 2023
• Journal: Renewable Energy
• Print ISSN: 0960-1481
• Electronic ISSN: 1879-0682
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 216
• Article Number: 118952
• DOI: https://doi.org/10.1016/j.renene.2023.118952
• Keywords: Solar energy, Forecasting, Computer vision, Deep learning, Vision Transformer, Sky images
• Public URL: https://nottingham-repository.worktribe.com/output/22436361
• Publisher URL: https://www.sciencedirect.com/science/article/pii/S0960148123008583