Black Box Search Space Profiling for Accelerator-Aware Neural Architecture Search
2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC)(2020)
摘要
Neural Architecture Search (NAS) is a promising approach to discover good neural network architectures for given applications. Among the three basic components in a NAS system (search space, search strategy, and evaluation), prior work mainly focused on the development of different search strategies and evaluation methods. As most of the previous hardware-aware search space designs aimed at CPUs and GPUs, it still remains a challenge to design a suitable search space for Deep Neural Network (DNN) accelerators. Besides, the architectures and compilers of DNN accelerators vary greatly, so it is quite difficult to get a unified and accurate evaluation of the latency of DNN across different platforms. To address these issues, we propose a black box profiling-based search space tuning method and further improve the latency evaluation by introducing a layer adaptive latency correction method. Used as the first stage in our general accelerator-aware NAS pipeline, our proposed methods could provide a smaller and dynamic search space with a controllable trade-off between accuracy and latency for DNN accelerators. Experimental results on CIFAR-10 and ImageNet demonstrate our search space is effective with up to 12.7% improvement in accuracy and 2.2× reduction of latency, and also efficient by reducing the search time and GPU memory up to 4.35× and 6.25×, respectively.
更多查看译文
关键词
hardware-aware search space designs,accelerator-aware neural architecture search,black box search space profiling,dynamic search space,smaller search space,general accelerator-aware NAS pipeline,layer adaptive latency correction method,latency evaluation,black box profiling-based search space tuning method,unified evaluation,compilers,deep neural network accelerators,search space,evaluation methods,search strategy,NAS system,neural network architectures,search time,DNN accelerators
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络