A Multi-Objective Evolutionary Approach Based on Graph-in-Graph for Neural Architecture Search of Convolutional Neural Networks

Xue, Yu; Jiang, Pengcheng; Neri, Ferrante; Liang, Jiayu

Abstract

With the development of deep learning, the design of an appropriate network structure becomes fundamental. In recent years, the successful practice of Neural Architecture Search (NAS) has indicated that an automated design of the network structure can efficiently replace the design performed by human experts. Most NAS algorithms make the assumption that the overall structure of the network is linear and focus solely on accuracy to assess the performance of candidate networks. This paper introduces a novel NAS algorithm based on a multi-objective modeling of the network design problem to design accurate Convolutional Neural Networks (CNNs) with a small structure. The proposed algorithm makes use of a graph-based representation of the solutions which enables a high flexibility in the automatic design. Furthermore, the proposed algorithm includes novel ad-hoc crossover and mutation operators. We also propose a mechanism to accelerate the evaluation of the candidate solutions. Experimental results demonstrate that the proposed NAS approach can design accurate neural networks with limited size.

• Journal Article Type: Article
• Acceptance Date: May 29, 2021
• Online Publication Date: Jul 24, 2021
• Publication Date: 2021-09
• Deposit Date: May 29, 2021
• Publicly Available Date: Jul 25, 2022
• Journal: International Journal of Neural Systems
• Print ISSN: 0129-0657
• Electronic ISSN: 1793-6462
• Peer Reviewed: Peer Reviewed
• Volume: 31
• Issue: 09
• Article Number: 2150035
• DOI: https://doi.org/10.1142/S0129065721500350
• Keywords: Deep Learning; Neural Architecture Search; Multi-objective Optimization; Genetic Algorithm 14
• Public URL: https://nottingham-repository.worktribe.com/output/5614611
• Publisher URL: https://www.worldscientific.com/doi/10.1142/S0129065721500350
• Additional Information: Electronic version of an article published as International Journal of Neural Systems, Volume 31, Issue 09, 2021, https://doi.org/10.1142/S0129065721500350 © 2021 World Scientific Publishing Company. https://www.worldscientific.com/doi/10.1142/S0129065721500350