Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search

Ruijun Xu
Ruijun Xu
Hailong Ma
Hailong Ma

arXiv: Neural and Evolutionary Computing, Volume abs/1901.01074, 2019.

Cited by: 2|Bibtex|Views101|Links
EI
Keywords:
Super-resolution domainmulti objectivereinforced learning methodsgenetic algorithmsuper resolutionMore(12+)
Weibo:
We propose a multi-objective reinforced evolution algorithm in mobile neural architecture search, which seeks a better trade-off among various competing objectives

Abstract:

Fabricating neural models for a wide range of mobile devices demands for a specific design of networks due to highly constrained resources. Both evolution algorithms (EA) and reinforced learning methods (RL) have been dedicated to solve neural architecture search problems. However, these combinations usually concentrate on a single object...More

Code:

Data:

0
Introduction
  • Automated neural architecture search has witnessed a victory versus human experts, confirming itself as the generation paradigm of architectural engineering.
  • Applying multi-objective NAS for the first time in Super-resolution domain other than common classification tasks and the results dominate some of the state-of-the-art deep-learning based SR methods, with the best models aligned near the Pareto front in a single run, 4.
  • It is necessary to embed this search problem in a real multi-objective context, where a series of models is found along the Pareto front of multiobjectives like accuracy, computational cost or inference time, number of parameters, etc.
Highlights
  • Automated neural architecture search has witnessed a victory versus human experts, confirming itself as the generation paradigm of architectural engineering
  • Our approach differs from previous works by: 1. inheriting the advantages from both NSGA-II and reinforcement learning to perform multi-objective NAS while overcoming the drawbacks from each method, 2. construction of cell-based search space to allow for genetic crossover and mutation, hierarchical organization of reinforced mutator with a natural mutation to assure the convergence and to speed up genetic selection, 3. applying multi-objective NAS for the first time in Super-resolution domain other than common classification tasks and the results dominate some of the state-of-the-art deep-learning based SR methods, with the best models aligned near the Pareto front in a single run, 4. involving minimum human expertise in model designing as an early guide, and imposing some practical constraints to obtain feasible solutions
  • MoreMNAS contains three basic components: cell-based search space, a model generation controller regarding multi-objective based on NSGA-II and an evaluator to return multi-reward feedback for each child model
  • We propose a multi-objective reinforced evolution algorithm in mobile neural architecture search, which seeks a better trade-off among various competing objectives
  • Our work is the first approach to perform multi-objective neural architecture search by combining NSGA-II and reinforcement learning
  • Our method is evaluated in the super-resolution domain
Results
  • As shown in Figure 2, MoreMNAS contains three basic components: cell-based search space, a model generation controller regarding multi-objective based on NSGA-II and an evaluator to return multi-reward feedback for each child model.
  • Cell-based Crossover Hierarchical Mutation train multireward searches on the basis of wild basic operators and makes use of some good cells already discovered (Zoph et al, 2017; Tan et al, 2018).
  • This approach constructs a one-toone mapping between the cell code and neural architectures, possessing an inherent advantage over NSGA-Net, whose space design involves a many-to-one mapping that further repetition removal steps must be taken to avoid meaningless training labors (Lu et al, 2018) 2.
  • Model Generation based on NSGA-II with Cell-level Crossover and Hierarchical Mutation
  • The authors initialize each individual by randomly selecting basic operators for each cell from search space and
  • Pure evolution based NAS algorithms seldom makes good use of the knowledge from training the generated models, which can be utilized to guide further search (Real et al, 2018; Cheng et al, 2018).
  • The authors take advantage of learned knowledge from training generated models and their scores of objectives based on a reinforced mutation controller, which is designed to distil meaningful experience to evaluate model performance.
  • Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search Table 2.
  • The authors propose a multi-objective reinforced evolution algorithm in mobile neural architecture search, which seeks a better trade-off among various competing objectives.
  • It has three obvious advantages: no recession of models during the whole process, good exploitation from reinforced mutation, a better balance of different objectives based on NSGA-II and Roulette-wheel selection.
Conclusion
  • The authors' work is the first approach to perform multi-objective neural architecture search by combining NSGA-II and reinforcement learning.
  • The authors generate several light-weight models that are very competitive, sometimes dominating human expert designed ones such as SRCNN, VDSR across several critical objectives: PSNR, multi-adds, and the number of parameters.
  • The authors' algorithm can be applied in other situations, limited to a mobile setting
Summary
  • Automated neural architecture search has witnessed a victory versus human experts, confirming itself as the generation paradigm of architectural engineering.
  • Applying multi-objective NAS for the first time in Super-resolution domain other than common classification tasks and the results dominate some of the state-of-the-art deep-learning based SR methods, with the best models aligned near the Pareto front in a single run, 4.
  • It is necessary to embed this search problem in a real multi-objective context, where a series of models is found along the Pareto front of multiobjectives like accuracy, computational cost or inference time, number of parameters, etc.
  • As shown in Figure 2, MoreMNAS contains three basic components: cell-based search space, a model generation controller regarding multi-objective based on NSGA-II and an evaluator to return multi-reward feedback for each child model.
  • Cell-based Crossover Hierarchical Mutation train multireward searches on the basis of wild basic operators and makes use of some good cells already discovered (Zoph et al, 2017; Tan et al, 2018).
  • This approach constructs a one-toone mapping between the cell code and neural architectures, possessing an inherent advantage over NSGA-Net, whose space design involves a many-to-one mapping that further repetition removal steps must be taken to avoid meaningless training labors (Lu et al, 2018) 2.
  • Model Generation based on NSGA-II with Cell-level Crossover and Hierarchical Mutation
  • The authors initialize each individual by randomly selecting basic operators for each cell from search space and
  • Pure evolution based NAS algorithms seldom makes good use of the knowledge from training the generated models, which can be utilized to guide further search (Real et al, 2018; Cheng et al, 2018).
  • The authors take advantage of learned knowledge from training generated models and their scores of objectives based on a reinforced mutation controller, which is designed to distil meaningful experience to evaluate model performance.
  • Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search Table 2.
  • The authors propose a multi-objective reinforced evolution algorithm in mobile neural architecture search, which seeks a better trade-off among various competing objectives.
  • It has three obvious advantages: no recession of models during the whole process, good exploitation from reinforced mutation, a better balance of different objectives based on NSGA-II and Roulette-wheel selection.
  • The authors' work is the first approach to perform multi-objective neural architecture search by combining NSGA-II and reinforcement learning.
  • The authors generate several light-weight models that are very competitive, sometimes dominating human expert designed ones such as SRCNN, VDSR across several critical objectives: PSNR, multi-adds, and the number of parameters.
  • The authors' algorithm can be applied in other situations, limited to a mobile setting
Tables
  • Table1: Hyper-parameters for the whole pipeline
  • Table2: Comparisons with the state-of-the-art methods based on ×2 super-resolution task†
Download tables as Excel
Related work
  • 2.1. Single-Objective Oriented NAS

    The majority of early NAS approaches fall in this category, where the validation accuracy of the model is the sole objective.

    2.1.1. REINFORCEMENT LEARNING-BASED APPROACHES

    Excessive research works have applied reinforcement learning in neural architecture search as aforementioned. They can be loosely divided into two genres according to the type of RL techniques: Q-learning and Policy Gradient.

    For Q-learning based methods like MetaQNN (Baker et al, 2017), a learning agent interacts with the environment by choosing CNN architectures from finite search space. It stores validation accuracy and architecture description in replay memory to be periodically sampled. The agent is enabled with -greedy strategy to balance exploration and exploitation.
Study subjects and analysis
individuals: 56
Setup and Implementation Details. In our experiment, the whole pipeline contains 200 generations, during each generation 56 individuals get spawned, i.e., there are 11200 models generated in total. Other hyperparameters is listed in Table 1

Reference
  • Ahn, N., Kang, B., and Sohn, K.-A. Fast, accurate, and, lightweight super-resolution with cascading residual network. arXiv preprint arXiv:1803.08664, 2018.
    Findings
  • Baker, B., Gupta, O., Naik, N., and Raskar, R. Designing neural network architectures using reinforcement learning. International Conference on Learning Representations, 2017.
    Google ScholarLocate open access versionFindings
  • Chen, Y., Zhang, Q., Huang, C., Mu, L., Meng, G., and Wang, X. Reinforced evolutionary neural architecture search. arXiv preprint arXiv:1808.00193, 2018.
    Findings
  • Cheng, A.-C., Dong, J.-D., Hsu, C.-H., Chang, S.-H., Sun, M., Chang, S.-C., Pan, J.-Y., Chen, Y.-T., Wei, W., and Juan, D.-C. Searching toward pareto-optimal deviceaware neural architectures. In Proceedings of the International Conference on Computer-Aided Design, pp. 136. ACM, 2018.
    Google ScholarLocate open access versionFindings
  • Chu, X. Policy optimization with penalized point probability distance: An alternative to proximal policy optimization. arXiv preprint arXiv:1807.00442, 2018.
    Findings
  • Chu, X. and Yu, X. Improved crowding distance for nsga-ii. arXiv preprint arXiv:1811.12667, 2018.
    Findings
  • Deb, K., Pratap, A., Agarwal, S., and Meyarivan, T. A fast and elitist multiobjective genetic algorithm: Nsgaii. IEEE transactions on evolutionary computation, 6(2): 182–197, 2002.
    Google ScholarLocate open access versionFindings
  • Dong, J.-D., Cheng, A.-C., Juan, D.-C., Wei, W., and Sun, M. Ppp-net: Platform-aware progressive search for pareto-optimal neural architectures. 201URL https://openreview.net/forum?id=B1NT3TAIM.
    Findings
  • Elsken, T., Metzen, J. H., and Hutter, F. Efficient multiobjective neural architecture search via lamarckian evolution. 2018.
    Google ScholarFindings
  • Gwiazda, T. D. Crossover for single-objective numerical optimization problems, volume 1. Tomasz Gwiazda, 2006.
    Google ScholarFindings
  • Haris, M., Shakhnarovich, G., and Ukita, N. Deep backprojection networks for super-resolution. In Conference on Computer Vision and Pattern Recognition, 2018.
    Google ScholarLocate open access versionFindings
  • He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
    Google ScholarLocate open access versionFindings
  • Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
    Google ScholarLocate open access versionFindings
  • Holland, J. H. Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence, 1975.
    Google ScholarFindings
  • Hore, A. and Ziou, D. Image quality metrics: Psnr vs. ssim. In Pattern recognition (icpr), 2010 20th international conference on, pp. 2366–2369. IEEE, 2010.
    Google ScholarLocate open access versionFindings
  • Hsu, C.-H., Chang, S.-H., Juan, D.-C., Pan, J.-Y., Chen, Y.-T., Wei, W., and Chang, S.-C. Monas: Multi-objective neural architecture search using reinforcement learning. arXiv preprint arXiv:1806.10332, 2018.
    Findings
  • Kim, J., Kwon Lee, J., and Mu Lee, K. Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1646–1654, 2016.
    Google ScholarLocate open access versionFindings
  • Kim, Y.-H., Reddy, B., Yun, S., and Seo, C. Nemo: Neuroevolution with multiobjective optimization of deep neural network for speed and accuracy. 2017.
    Google ScholarFindings
  • Lim, B., Son, S., Kim, H., Nah, S., and Lee, K. M. Enhanced deep residual networks for single image super-resolution. In The IEEE conference on computer vision and pattern recognition (CVPR) workshops, volume 1, pp. 4, 2017.
    Google ScholarLocate open access versionFindings
  • Lipowski, A. and Lipowska, D. Roulette-wheel selection via stochastic acceptance. Physica A: Statistical Mechanics and its Applications, 391(6):2193–2196, 2012.
    Google ScholarLocate open access versionFindings
  • Liu, C., Zoph, B., Shlens, J., Hua, W., Li, L.-J., Fei-Fei, L., Yuille, A., Huang, J., and Murphy, K. Progressive neural architecture search. arXiv preprint arXiv:1712.00559, 2017a.
    Findings
  • Liu, H., Simonyan, K., Vinyals, O., Fernando, C., and Kavukcuoglu, K. Hierarchical representations for efficient architecture search. arXiv preprint arXiv:1711.00436, 2017b.
    Findings
  • Lu, Z., Whalen, I., Boddeti, V., Dhebar, Y., Deb, K., Goodman, E., and Banzhaf, W. Nsga-net: A multi-objective genetic algorithm for neural architecture search. arXiv preprint arXiv:1810.03522, 2018.
    Findings
  • Ma, N., Zhang, X., Zheng, H.-T., and Sun, J. Shufflenet v2: Practical guidelines for efficient cnn architecture design. arXiv preprint arXiv:1807.11164, 1, 2018.
    Findings
  • Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. Human-level control through deep reinforcement learning. Nature, 518(7540): 529, 2015.
    Google ScholarLocate open access versionFindings
  • Negrinho, R. and Gordon, G. Deeparchitect: Automatically designing and training deep architectures. arXiv preprint arXiv:1704.08792, 2017.
    Findings
  • Pham, H., Guan, M. Y., Zoph, B., Le, Q. V., and Dean, J. Efficient neural architecture search via parameter sharing. arXiv preprint arXiv:1802.03268, 2018.
    Findings
  • Real, E., Aggarwal, A., Huang, Y., and Le, Q. V. Regularized evolution for image classifier architecture search. arXiv preprint arXiv:1802.01548, 2018.
    Findings
  • Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510– 4520, 2018.
    Google ScholarLocate open access versionFindings
  • Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
    Findings
  • Stanley, K. O. and Miikkulainen, R. Evolving neural networks through augmenting topologies. Evolutionary computation, 10(2):99–127, 2002.
    Google ScholarLocate open access versionFindings
  • Sutton, R. S. and Barto, A. G. Reinforcement learning: An introduction. MIT press, 2018.
    Google ScholarFindings
  • Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1–9, 2015.
    Google ScholarLocate open access versionFindings
  • Tai, Y., Yang, J., and Liu, X. Image super-resolution via deep recursive residual network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, volume 1, pp. 5, 2017.
    Google ScholarLocate open access versionFindings
  • Tan, M., Chen, B., Pang, R., Vasudevan, V., and Le, Q. V. Mnasnet: Platform-aware neural architecture search for mobile. arXiv preprint arXiv:1807.11626, 2018.
    Findings
  • Timofte, R., Agustsson, E., Van Gool, L., Yang, M.-H., Zhang, L., Lim, B., Son, S., Kim, H., Nah, S., Lee, K. M., et al. Ntire 2017 challenge on single image superresolution: Methods and results. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2017 IEEE Conference on, pp. 1110–1121. IEEE, 2017.
    Google ScholarLocate open access versionFindings
  • Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004.
    Google ScholarLocate open access versionFindings
  • Xie, L. and Yuille, A. Genetic cnn. pp. 1388–1397, 10 2017. doi: 10.1109/ICCV.2017.154.
    Findings
  • Yu, X. and Gen, M. Introduction to evolutionary algorithms. Springer Science & Business Media, 2010.
    Google ScholarFindings
  • Zoph, B. and Le, Q. V. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578, 2016.
    Findings
  • Zoph, B., Vasudevan, V., Shlens, J., and Le, Q. V. Learning transferable architectures for scalable image recognition. arXiv preprint arXiv:1707.07012, 2(6), 2017.
    Findings
Your rating :
0

 

Tags
Comments