Memory-Efficient Hierarchical Neural Architecture Search for Image Denoising

    CVPR 2020, 2020.

    Cited by: 1|Bibtex|Views25|Links
    Keywords:
    architecture searchconvolutional neural networksneural architecture searchreinforcement learningclean imageMore(10+)
    Wei bo:
    We have proposed HiNAS, an memoryefficient hierarchical architecture search algorithm for the low-level image restoration task image denoising

    Abstract:

    Recently, neural architecture search (NAS) methods have attracted much attention and outperformed manually designed architectures on a few high-level vision tasks. In this paper, we propose HiNAS (Hierarchical NAS), an effort towards employing NAS to automatically design effective neural network architectures for image denoising. HiNAS ad...More

    Code:

    Data:

    0
    Introduction
    • Single image denoising is an important task in low-level computer vision, which restores a clean image from a noisy one.
    • Traditional image denoising methods generally focus on modeling natural image priors and use the priors to restore the clean image, including sparse models [6, 27], Markov random field models [14], etc.
    • One drawback of these methods is that most of them involve a complex opti-.
    • Discovering state-of-the-art neural network architectures requires substantial efforts
    Highlights
    • Single image denoising is an important task in low-level computer vision, which restores a clean image from a noisy one
    • Owing to the fact that noise corruption always occurs in the image sensing process and may degrade the visual quality of collected images, image denoising is needed for various computer vision tasks [3]
    • Traditional image denoising methods generally focus on modeling natural image priors and use the priors to restore the clean image, including sparse models [6, 27], Markov random field models [14], etc
    • Based on gradient based search algorithms, we propose a memory-efficient hierarchical neural architecture search approach for image denoising, termed HiNAS
    • We have proposed HiNAS, an memoryefficient hierarchical architecture search algorithm for the low-level image restoration task image denoising
    Methods
    • Conventional convolution, dilated convolution and skip connection
    • The authors believe that these results prove that HiNAS is able to select proper operators.
    • The authors verify if HiNAS improves the accuracy by designing a proper architecture or by integrating various branch structures and convolution operations.
    • The authors mainly focus on comparing the HiNAS with E-CAE, because both them are proposed for searching for architectures for the task of denoising on BSD500.
    • Table 5 shows that N3Net and HiNAS beat other models by a clear margin.
    • When the noise level σ is set to 30, the SSIM of NLRN is slightly higher (0.002) than that of the HiNAS, but the PSNR of NLRN is much lower than that of HiNAS
    Results
    • The authors verify if HiNAS improves the accuracy by designing a proper architecture or by integrating various branch structures and convolution operations.
    • The authors' proposed HiNAS achieves the best performance when σ is set to 50 and 70
    Conclusion
    • The authors have proposed HiNAS, an memoryefficient hierarchical architecture search algorithm for the low-level image restoration task image denoising.
    • HiNAS adopts differentiable architecture search algorithms and a cell sharing strategy.
    • It is both memory and computation efficient, taking only about 4.5 hours to search using a single GPU.
    • The authors believe that the proposed method can be applied to many other low-level image processing tasks
    Summary
    • Introduction:

      Single image denoising is an important task in low-level computer vision, which restores a clean image from a noisy one.
    • Traditional image denoising methods generally focus on modeling natural image priors and use the priors to restore the clean image, including sparse models [6, 27], Markov random field models [14], etc.
    • One drawback of these methods is that most of them involve a complex opti-.
    • Discovering state-of-the-art neural network architectures requires substantial efforts
    • Methods:

      Conventional convolution, dilated convolution and skip connection
    • The authors believe that these results prove that HiNAS is able to select proper operators.
    • The authors verify if HiNAS improves the accuracy by designing a proper architecture or by integrating various branch structures and convolution operations.
    • The authors mainly focus on comparing the HiNAS with E-CAE, because both them are proposed for searching for architectures for the task of denoising on BSD500.
    • Table 5 shows that N3Net and HiNAS beat other models by a clear margin.
    • When the noise level σ is set to 30, the SSIM of NLRN is slightly higher (0.002) than that of the HiNAS, but the PSNR of NLRN is much lower than that of HiNAS
    • Results:

      The authors verify if HiNAS improves the accuracy by designing a proper architecture or by integrating various branch structures and convolution operations.
    • The authors' proposed HiNAS achieves the best performance when σ is set to 50 and 70
    • Conclusion:

      The authors have proposed HiNAS, an memoryefficient hierarchical architecture search algorithm for the low-level image restoration task image denoising.
    • HiNAS adopts differentiable architecture search algorithms and a cell sharing strategy.
    • It is both memory and computation efficient, taking only about 4.5 hours to search using a single GPU.
    • The authors believe that the proposed method can be applied to many other low-level image processing tasks
    Tables
    • Table1: Comparisons of different search settings
    • Table2: Ablation study on BSD500. HiNAS∗ is trained with single loss MSE and HiNAS∗∗ is trained with the combination loss MSE and lssim
    • Table3: Architecture analysis
    • Table4: Comparisons with E-CAE on BSD500
    • Table5: Denoising experiments. Comparisons with state-of-the-arts on the BSD500 dataset. We show our results in the last row. Time cost means GPU-seconds for inference on the 200 images from the test set of BSD500 using one single GTX 980 graphic card
    • Table6: Denoising results on SIM1800
    • Table7: De-raining results on Rain800. With a GTX 980 graphic card, RESCAN and HiNAS respectively cost 44.35, 21.80 GPU-seconds for inference on the test set of Rain800
    Download tables as Excel
    Related work
    • CNNs for image denoising. To date, due to the popularity of convolutional neural networks (CNNs), image denoising algorithms have achieved a significant performance boost. Recent network models such as DnCNN [42] and IrCNN [43] predict the residue presented in the image instead of the denoised image, showing promising performance. Lately, FFDNet [44] attempts to address spatially varying noise by appending noise level maps to the input of DnCNN. NLRN [19] incorporates non-local operations into a recurrent neural network (RNN) for image restoration. N3Net [36] formulates a differentiable version of nearest neighbor search to further improve DnCNN. DuRN-P [23] proposes a new style of residual connection, where two residual connections are employed to exploit the potential of paired operations. Some algorithms focus on denoising for real-noisy images. CBDNet [11] uses a simulated camera pipeline to supplement real training data. Similar work in [13] proposes a camera simulator that aims to accurately simulate the degradation and noise transformation performed by camera pipelines. Network architecture search (NAS). NAS aims to design automated approaches for discovering high-performance neural architectures such that the procedure of tedious and heuristic manual design of neural architectures can be eliminated from the deep learning pipeline. Early attempts employ evolutionary algorithms (EAs) for optimizing neural architectures and parameters. The best architecture may be obtained by iteratively mutating a population of candidate architectures [20]. An alternative to EA is to use reinforcement learning (RL) techniques, e.g., policy gradients [48, 37] and Q-learning [45], to train a recurrent neural network that acts as a meta-controller to generate potential architectures—typically encoded as sequences—by exploring a predefined search space. However, EA and RL based methods are inefficient in search, often requiring a large amount of computations. Speed-up techniques are therefore proposed to remedy this issue. Exemplar works include hyper-networks [41], network morphism [7] and shared weights [31].
    Funding
    • Shen’s participation was in part supported by the ARC Grant “Deep learning that scales”
    Reference
    • Vladimir Bychkovsky, Sylvain Paris, Eric Chan, and Fredo Durand. Learning photographic global tonal adjustment with a database of input/output image pairs. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 97–104. IEEE, 2015
      Google ScholarLocate open access versionFindings
    • Han Cai, Ligeng Zhu, and Song Han. Proxylessnas: Direct neural architecture search on target task and hardware. arXiv: Comp. Res. Repository, abs/1812.00332, 2018. 2
      Google ScholarLocate open access versionFindings
    • Priyam Chatterjee and Peyman Milanfar. Is denoising dead? TIP, 19(4):895–911, 2009. 1
      Google ScholarLocate open access versionFindings
    • Xiangxiang Chu, Bo Zhang, Hailong Ma, Ruijun Xu, Jixiang Li, and Qingyuan Li. Fast, accurate and lightweight superresolution with neural architecture search. arXiv: Comp. Res. Repository, abs/1901.07261, 2019. 2, 7
      Google ScholarLocate open access versionFindings
    • K Dabov, A Foi, V Katkovnik, and K Egiazarian. Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans. Image Process., 16(8):2080–2095, 2007. 1, 7
      Google ScholarLocate open access versionFindings
    • Weisheng Dong, Lei Zhang, Guangming Shi, and Xin Li. Nonlocally centralized sparse representation for image restoration. TIP, 22(4):1620–1630, 2012. 1
      Google ScholarLocate open access versionFindings
    • Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Efficient multi-objective neural architecture search via lamarckian evolution. arXiv: Comp. Res. Repository, abs/1804.09081, 2018. 2
      Google ScholarLocate open access versionFindings
    • Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley. Removing rain from single images via a deep detail network. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 3855–3863, 2017. 8
      Google ScholarLocate open access versionFindings
    • Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le. Nas-fpn: Learning scalable feature pyramid architecture for object detection. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 7036–7045, 2011
      Google ScholarLocate open access versionFindings
    • Shuhang Gu, Lei Zhang, Wangmeng Zuo, and Xiangchu Feng. Weighted nuclear norm minimization with application to image denoising. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 2862–2869, 2014. 1, 7
      Google ScholarLocate open access versionFindings
    • Shi Guo, Zifei Yan, Kai Zhang, Wangmeng Zuo, and Lei Zhang. Toward convolutional blind denoising of real photographs. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 1712–1722, 2019. 2
      Google ScholarLocate open access versionFindings
    • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference on computer vision, pages 630–645. Springer, 2016. 2
      Google ScholarLocate open access versionFindings
    • Ronnachai Jaroensri, Camille Biscarrat, Miika Aittala, and Fredo Durand. Generating training data for denoising real rgb images via camera pipeline simulation. arXiv: Comp. Res. Repository, abs/1904.08825, 2019. 2, 5
      Google ScholarLocate open access versionFindings
    • Xiangyang Lan, Stefan Roth, Daniel Huttenlocher, and Michael J Black. Efficient belief propagation with learned higher-order markov random fields. In Proc. Eur. Conf. Comp. Vis., pages 269–282. Springer, 2006. 1
      Google ScholarLocate open access versionFindings
    • Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In Proc. Eur. Conf. Comp. Vis., pages 254–269, 2018. 3, 8
      Google ScholarLocate open access versionFindings
    • Yu Li, Robby T Tan, Xiaojie Guo, Jiangbo Lu, and Michael S Brown. Rain streak removal using layer priors. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 2736–2744, 208
      Google ScholarLocate open access versionFindings
    • Hanwen Liang, Shifeng Zhang, Jiacheng Sun, Xingqiu He, Weiran Huang, Kechen Zhuang, and Zhenguo Li. Darts+: Improved differentiable architecture search with early stopping. arXiv preprint arXiv:1909.06035, 2019. 5
      Findings
    • Chenxi Liu, Liang-Chieh Chen, Florian Schroff, Hartwig Adam, Wei Hua, Alan L Yuille, and Li Fei-Fei. Autodeeplab: Hierarchical neural architecture search for semantic image segmentation. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 82–92, 2019. 1, 2
      Google ScholarLocate open access versionFindings
    • Ding Liu, Bihan Wen, Yuchen Fan, Chen Change Loy, and Thomas S Huang. Non-local recurrent network for image restoration. In Proc. Advances in Neural Inf. Process. Syst., pages 1673–1682, 2018. 2, 3, 5, 7
      Google ScholarLocate open access versionFindings
    • Hanxiao Liu, Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, and Koray Kavukcuoglu. Hierarchical representations for efficient architecture search. arXiv: Comp. Res. Repository, abs/1711.00436, 2017. 2
      Google ScholarLocate open access versionFindings
    • Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv: Comp. Res. Repository, abs/1806.09055, 2018. 2
      Google ScholarLocate open access versionFindings
    • Peng Liu, Mohammad D El Basha, Yangjunyi Li, Yao Xiao, Pina C Sanelli, and Ruogu Fang. Deep evolutionary networks with expedited genetic algorithms for medical image denoising. Medical image analysis, 54:306–315, 2019. 7
      Google ScholarLocate open access versionFindings
    • Xing Liu, Masanori Suganuma, Zhun Sun, and Takayuki Okatani. Dual residual networks leveraging the potential of paired operations for image restoration. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 7007–7016, 2019. 1, 2, 5, 7
      Google ScholarLocate open access versionFindings
    • Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. In Proc. Int. Conf. Learn. Representations, 2017. 5
      Google ScholarLocate open access versionFindings
    • Yu Luo, Yong Xu, and Hui Ji. Removing rain from a single image via discriminative sparse coding. In Proc. IEEE Int. Conf. Comp. Vis., pages 3397–3405, 2015. 8
      Google ScholarLocate open access versionFindings
    • Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet v2: Practical guidelines for efficient cnn architecture design. In Proc. Eur. Conf. Comp. Vis., pages 116–131, 2018. 4, 6
      Google ScholarLocate open access versionFindings
    • Julien Mairal, Francis R Bach, Jean Ponce, Guillermo Sapiro, and Andrew Zisserman. Non-local sparse models for image restoration. In Proc. IEEE Int. Conf. Comp. Vis., volume 29, pages 54–62. Citeseer, 2009. 1
      Google ScholarLocate open access versionFindings
    • Xiaojiao Mao, Chunhua Shen, and Yu-Bin Yang. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Proc. Advances in Neural Inf. Process. Syst., pages 2802–2810, 2016. 1, 5, 7
      Google ScholarLocate open access versionFindings
    • David Martin, Charless Fowlkes, Doron Tal, Jitendra Malik, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. IEEE Int. Conf. Comp. Vis., pages 416–423, 2001. 5
      Google ScholarLocate open access versionFindings
    • Vladimir Nekrasov, Hao Chen, Chunhua Shen, and Ian Reid. Fast neural architecture search of compact semantic segmentation models via auxiliary cells. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 9126–9135, 2019. 1
      Google ScholarLocate open access versionFindings
    • Hieu Pham, Melody Y Guan, Barret Zoph, Quoc V Le, and Jeff Dean. Efficient neural architecture search via parameter sharing. arXiv: Comp. Res. Repository, abs/1802.03268, 2018. 2
      Google ScholarLocate open access versionFindings
    • Tobias Plotz and Stefan Roth. Neural nearest neighbors networks. In Proc. Advances in Neural Inf. Process. Syst., pages 1087–1098, 2018. 3, 6, 7
      Google ScholarLocate open access versionFindings
    • Masanori Suganuma, Mete Ozay, and Takayuki Okatani. Exploiting the potential of standard convolutional autoencoders for image restoration by evolutionary search. In Proc. Int. Conf. Mach. Learn., 2018. 2, 7
      Google ScholarLocate open access versionFindings
    • Ying Tai, Jian Yang, Xiaoming Liu, and Chunyan Xu. Memnet: A persistent memory network for image restoration. In Proc. IEEE Int. Conf. Comp. Vis., pages 4539–4547, 2017. 1, 5, 7
      Google ScholarLocate open access versionFindings
    • Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V Le. Mnasnet: Platform-aware neural architecture search for mobile. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 2820– 2828, 2019. 4
      Google ScholarLocate open access versionFindings
    • Stefan Roth Tobias Pltz. Neural nearest neighbors networks. In Proc. Advances in Neural Inf. Process. Syst., pages 1673– 1682, 2018. 1, 2
      Google ScholarLocate open access versionFindings
    • Ning Wang, Yang Gao, Hao Chen, Peng Wang, Zhi Tian, and Chunhua Shen. NAS-FCOS: Fast neural architecture search for object detection. arXiv: Comp. Res. Repository, abs/1906.04423, 2019. 1, 2
      Google ScholarLocate open access versionFindings
    • Zhou Wang, Alan Bovik, Hamid Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4):600–612, 2004. 4, 5
      Google ScholarLocate open access versionFindings
    • Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 10734– 10742, 2019. 4
      Google ScholarLocate open access versionFindings
    • Wenhan Yang, Robby T Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan. Deep joint rain detection and removal from a single image. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 1357–1366, 2017. 8
      Google ScholarLocate open access versionFindings
    • Chris Zhang, Mengye Ren, and Raquel Urtasun. Graph hypernetworks for neural architecture search. arXiv: Comp. Res. Repository, abs/1810.05749, 2018. 2
      Google ScholarLocate open access versionFindings
    • Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans. Image Process., 26(7):3142–3155, 2017. 2
      Google ScholarLocate open access versionFindings
    • Kai Zhang, Wangmeng Zuo, Shuhang Gu, and Lei Zhang. Learning deep cnn denoiser prior for image restoration. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 3929– 3938, 2017. 2
      Google ScholarLocate open access versionFindings
    • Kai Zhang, Wangmeng Zuo, and Lei Zhang. Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. IEEE Trans. Image Process., 27(9):4608–4622, 2018. 2
      Google ScholarLocate open access versionFindings
    • Zhao Zhong, Junjie Yan, Wei Wu, Jing Shao, and ChengLin Liu. Practical block-wise neural network architecture generation. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 2423–2432, 2018. 2
      Google ScholarLocate open access versionFindings
    • Xizhou Zhu, Han Hu, Stephen Lin, and Jifeng Dai. Deformable convnets v2: More deformable, better results. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 9308– 9316, 2019. 3
      Google ScholarLocate open access versionFindings
    • Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning. arXiv: Comp. Res. Repository, abs/1611.01578, 2016. 1
      Google ScholarLocate open access versionFindings
    • Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. Learning transferable architectures for scalable image recognition. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 8697–8710, 2018. 2
      Google ScholarLocate open access versionFindings
    Your rating :
    0

     

    Tags
    Comments