Skip to main content

Research Repository

Advanced Search

Deep Learning with Attention Mechanisms for Road Weather Detection

Samo, Madiha; Mafeni Mase, Jimiama Mosima; Figueredo, Grazziela

Deep Learning with Attention Mechanisms for Road Weather Detection Thumbnail


Authors

Madiha Samo

Jimiama Mosima Mafeni Mase



Abstract

There is great interest in automatically detecting road weather and understanding its impacts on the overall safety of the transport network. This can, for example, support road condition-based maintenance or even serve as detection systems that assist safe driving during adverse climate conditions. In computer vision, previous work has demonstrated the effectiveness of deep learning in predicting weather conditions from outdoor images. However, training deep learning models to accurately predict weather conditions using real-world road-facing images is difficult due to: (1) the simultaneous occurrence of multiple weather conditions; (2) imbalanced occurrence of weather conditions throughout the year; and (3) road idiosyncrasies, such as road layouts, illumination, and road objects, etc. In this paper, we explore the use of a focal loss function to force the learning process to focus on weather instances that are hard to learn with the objective of helping address data imbalances. In addition, we explore the attention mechanism for pixel-based dynamic weight adjustment to handle road idiosyncrasies using state-of-the-art vision transformer models. Experiments with a novel multi-label road weather dataset show that focal loss significantly increases the accuracy of computer vision approaches for imbalanced weather conditions. Furthermore, vision transformers outperform current state-of-the-art convolutional neural networks in predicting weather conditions with a validation accuracy of 92% and an F1-score of 81.22%, which is impressive considering the imbalanced nature of the dataset.

Citation

Samo, M., Mafeni Mase, J. M., & Figueredo, G. (2023). Deep Learning with Attention Mechanisms for Road Weather Detection. Sensors, 23(2), Article 798. https://doi.org/10.3390/s23020798

Journal Article Type Article
Acceptance Date Jan 6, 2023
Online Publication Date Jan 10, 2023
Publication Date 2023-01
Deposit Date Jan 20, 2023
Publicly Available Date Mar 29, 2024
Journal Sensors
Publisher MDPI AG
Peer Reviewed Peer Reviewed
Volume 23
Issue 2
Article Number 798
DOI https://doi.org/10.3390/s23020798
Keywords Computer vision; deep learning; image classification; loss functions; vision transformers; weather detection; autonomous vehicles
Public URL https://nottingham-repository.worktribe.com/output/15940388
Publisher URL https://www.mdpi.com/1424-8220/23/2/798
Additional Information This article belongs to the Special Issue Cyber-Physical Systems and Industry 4.0