Integrating aerial and street view images for urban land use classification

Cao, Rui; Zhu, Jiasong; Tu, Wei; Li, Qingquan; Cao, Jinzhou; Liu, Bozhi; Zhang, Qian; Qiu, Guoping

Abstract

Urban land use is key to rational urban planning and management. Traditional land use classification methods rely heavily on domain experts, which is both expensive and inefficient. In this paper, deep neural network-based approaches are presented to label urban land use at pixel level using high-resolution aerial images and ground-level street view images. We use a deep neural network to extract semantic features from sparsely distributed street view images and interpolate them in the spatial domain to match the spatial resolution of the aerial images, which are then fused together through a deep neural network for classifying land use categories. Our methods are tested on a large publicly available aerial and street view images dataset of New York City, and the results show that using aerial images alone can achieve relatively high classification accuracy, the ground-level street view images contain useful information for urban land use classification, and fusing street image features with aerial images can improve classification accuracy. Moreover, we present experimental studies to show that street view images add more values when the resolutions of the aerial images are lower, and we also present case studies to illustrate how street view images provide useful auxiliary information to aerial images to boost performances.

Citation

Cao, R., Zhu, J., Tu, W., Li, Q., Cao, J., Liu, B., Zhang, Q., & Qiu, G. (2018). Integrating aerial and street view images for urban land use classification. Remote Sensing, 10(10), Article 1553. https://doi.org/10.3390/rs10101553

• Journal Article Type: Article
• Acceptance Date: Sep 25, 2018
• Online Publication Date: Sep 27, 2018
• Publication Date: Oct 31, 2018
• Deposit Date: Jan 16, 2019
• Publicly Available Date: Jan 16, 2019
• Journal: Remote Sensing
• Electronic ISSN: 2072-4292
• Publisher: MDPI
• Peer Reviewed: Peer Reviewed
• Volume: 10
• Issue: 10
• Article Number: 1553
• DOI: https://doi.org/10.3390/rs10101553
• Keywords: Land use classification; Semantic segmentation; Aerial images; Street view images; Convolutional neural network (CNN); Deep learning; Data fusion
• Public URL: https://nottingham-repository.worktribe.com/output/1475628
• Publisher URL: https://www.mdpi.com/2072-4292/10/10/1553