ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2021, Vol. 58 ›› Issue (1): 22-33.doi: 10.7544/issn1000-1239.2021.20190851

• 人工智能 • 上一篇    下一篇

深度神经架构搜索综述

孟子尧1,2, 谷雪1,2, 梁艳春1,2, 许东3, 吴春国1   

  1. 1(符号计算与知识工程教育部重点实验室(吉林大学) 长春 130012);2(符号计算与知识工程教育部重点实验室珠海分实验室(吉林大学珠海学院) 广东珠海 519041);3(密苏里大学哥伦比亚分校电子工程与计算机科学系 美国密苏里州哥伦比亚 MO65211) (zy-meng@outlook.com)
  • 出版日期: 2021-01-01
  • 基金资助: 
    国家自然科学基金项目(61972174,61876069,61876207);吉林省重点研发项目(20180201045GX,20180201067GX);吉林省自然科学基金项目(20200201163JC);广东省科技计划项目(2020A0505100018);广东省应用基础研究重点项目(2018KZDXM076);广东省优势重点学科项目(2016GDYSZDXK036)

Deep Neural Architecture Search: A Survey

Meng Ziyao1,2, Gu Xue1,2, Liang Yanchun1,2, Xu Dong3, Wu Chunguo1   

  1. 1(Key Laboratory of Symbol Computation and Knowledge Engineering (Jilin University), Ministry of Education, Changchun 130012);2(Zhuhai Laboratory of Key Laboratory of Symbol Computation and Knowledge Engineering (Zhuhai College of Jilin University), Ministry of Education, Zhuhai, Guangdong 519041);3(Department of Electrical Engineering and Computer Science, University of Missouri-Columbia, Columbia, Missouri, USA MO65211)
  • Online: 2021-01-01
  • Supported by: 
    This work was supported by the National Natural Science Foundation of China (61972174, 61876069, 61876207), the Key Research and Development Project of Jilin Province (20180201045GX, 20180201067GX), the Natural Science Foundation of Jilin Province(20200201163JC), the Science and Technology Planning Project of Guangdong Province (2020A0505100018), the Guangdong Key-Project for Applied Fundamental Research (2018KZDXM076), and the Guangdong Premier Key-Discipline Enhancement Scheme (2016GDYSZDXK036).

摘要: 深度学习在图像、语音、文本等多种模态的数据任务上取得了优异的效果.然而,针对特定任务,人工设计网络需要花费大量的时间,并且需要设计者具有一定水平的专业知识和设计经验.面对如今日趋复杂的网络架构,仅依靠人工进行设计变得越来越复杂.基于此,借助算法自动地对神经网络进行架构的搜索成为了研究热点.神经架构搜索的方法涉及3个方面:搜索空间、搜索策略、性能评估策略.通过搜索策略在搜索空间中选择一个网络架构,借助性能评估策略对该网络架构进行评估,并将结果反馈给搜索策略指导搜索策略选择更好的网络架构,通过不断迭代得到最优的网络架构.为了更好地为读者提供一个快速了解神经网络架构搜索方法的导航地图,从搜索空间、搜索策略和性能评估策略3个方面对现有典型的神经架构搜索方法进行了梳理,总结讨论了近年来常见的架构搜索方法,并分析了各种方法的优势和不足.

关键词: 深度学习, 神经架构搜索, 搜索空间, 搜索策略, 性能评估

Abstract: Deep learning has achieved excellent results on data tasks with multiple modalities such as images, speech, and text. However, designing networks manually for specific tasks is time-consuming and requires a certain level of expertise and design experience from the designer. In the face of today’s increasingly complex network architectures, relying on manual design alone increasingly becomes complex. For this reason, automatic architecture search of neural networks with the help of algorithms has become a hot research topic. The approach of neural architecture search involves three aspects: search space, search strategy, and performance evaluation strategy. The search strategy samples a network architecture in the search space, evaluates the network architecture by a performance evaluation strategy, and feed-back the results to the search strategy to guide it to select a better network architecture, and obtains the optimal network architecture through continuous iterations. In order to better sort out the methods of neural architecture search, we summarize the common methods in recent years from search space, search strategy and performance evaluation strategy, and analyze their strengths and weaknesses.

Key words: deep learning, neural architecture search (NAS), search space, search strategy, perfor-mance evaluation

中图分类号: