AI helps you reading Science
AI generates interpretation videos
AI extracts and analyses the key points of the paper to generate videos automatically
AI parses the academic lineage of this thesis
AI extracts a summary of this paper
We proposed a deep convolutional network for inverse problems with a focus on biomedical imaging
Deep Convolutional Neural Network for Inverse Problems in Imaging.
IEEE Transactions on Image Processing, no. 9 (2017): 4509-4522
In this paper, we propose a novel deep convolutional neural network (CNN)-based algorithm for solving ill-posed inverse problems. Regularized iterative algorithms have emerged as the standard approach to ill-posed inverse problems in the past few decades. These methods produce excellent results, but can be challenging to deploy in practic...More
PPT (Upload PPT)
- Manuscript received November 4, 2016; revised April 10, 2017 and May 24, 2017; accepted May 24, 2017.
- The associate editor coordinating the review of this manuscript and approving it for publication was Prof.
- E. Froustey was with the Biomedical Imaging Group, École Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland.
- Froustey was with the Biomedical Imaging Group, École Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland
- He is with Dassault Aviation, 92210 Saint-Cloud, France
- O VER the past decades, iterative reconstruction methods have become the dominant approach to solving inverse
Manuscript received November 4, 2016; revised April 10, 2017 and May 24, 2017; accepted May 24, 2017
- Though the FBPConvNet algorithm is general, we focus here on sparse-view X-ray computed tomography reconstruction
- We compare the results of applying the TV and filtered back projection methods to the subsampled sinogram with the results of applying the FBPConvNet to the same. This type of sparse-view reconstruction is of particular interest for human imaging because, e.g., a twenty times reduction in the number of views corresponds to a twenty times reduction in the radiation dose received by the patient
- We proposed a deep convolutional network for inverse problems with a focus on biomedical imaging
- The structure of the convolutional neural network is based on U-net, with the addition of residual learning
- This approach was motivated by the convolutional structure of several biomedical inverse problems, including computed tomography, magnetic resonance imaging, and diffraction tomography
- The success of iterative methods consisting of filtering plus pointwise nonlinearities on normal-convolutional inverse problems suggests that CNNs may be a good fit for these problems as well.
- Based on this insight, the authors propose a new approach to these problems, which the authors call the FBPConvNet. B.
- There are several properties of this architecture that recommend it for the purposes
- The authors describe the experimental setup and results. Though the FBPConvNet algorithm is general, the authors focus here on sparse-view X-ray CT reconstruction.
- The authors compare FBPConvNet to FBP alone and a state-of-the-art iterative reconstruction method 
- This method solves a version of Eq (1) with the popular TV regularization via ADMM.
- The authors compare the results of applying the TV and FBP methods to the subsampled sinogram with the results of applying the FBPConvNet to the same
- This type of sparse-view reconstruction is of particular interest for human imaging because, e.g., a twenty times reduction in the number of views corresponds to a twenty times reduction in the radiation dose received by the patient.
- The experiments provide strong evidence for the feasibility of the FBPConvNet for sparse-view CT reconstruction.
- The conventional iterative algorithm with TV regularization outperformed the FBPConvNet in the ellipsoidal dataset, while the reverse was true for the biomedical and experimental datasets
- In these more-realistic datasets, the SNR improvement of the FBPConvNet came from its ability to preserve fine details in the images.
- This results suggests that CNNs are well-suited to this subclass of inverse problems
- Table1: COMPARISON OF SNR BETWEEN DIFFERENT RECONSTRUCTION ALGORITHMS FOR NUMERICAL ELLIPSOIDAL DATASET
- Table2: COMPARISON OF SNR BETWEEN DIFFERENT RECONSTRUCTION ALGORITHMS FOR BIOMEDICAL DATASET
- Table3: COMPARISON OF SNR BETWEEN DIFFERENT ARCHITECTURES FOR BIOMEDICAL DATASET
- Table4: COMPARISON OF SNR BETWEEN DIFFERENT RECONSTRUCTION ALGORITHMS FOR EXPERIMENTAL DATASET
- This work was supported in part by the European Union’s Horizon 2020 Framework Programme for Research and Innovation under Grant 665667 (call 2015), in part by the Center for Biomedical Imaging of the Geneva-Lausanne Universities and EPFL, in part by the European Research Council under Grant 692726 (H2020-ERC Project GlobalBioIm), and in part by the National Institute of Biomedical Imaging and Bioengineering under Grant EB017095 and under Grant EB017185
- L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Phys. D, Nonlinear Phenomena, vol. 60, nos. 1–4, pp. 259–268, 1992.
- A. Chambolle, “An algorithm for total variation minimization and applications,” J. Math. Imag. Vis., vol. 20, no. 1, pp. 89–97, 2004.
- S. G. Chang, B. Yu, and M. Vetterli, “Adaptive wavelet thresholding for image denoising and compression,” IEEE Trans. Image Process., vol. 9, no. 9, pp. 1532–1546, Sep. 2000.
- T. F. Chan and C.-K. Wong, “Total variation blind deconvolution,” IEEE Trans. Image Process., vol. 7, no. 3, pp. 370–375, Mar. 1998.
- D. Krishnan and R. Fergus, “Fast image deconvolution using hyperLaplacian priors,” in Proc. Adv. Neural Inf. Process. Syst., 2009, pp. 1033–1041.
- M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proc. 27th Annu. Conf. Comput. Graph. Interactive Tech., 2000, pp. 417–424.
- E. J. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489–509, Feb. 2006.
- T. Goldstein and S. Osher, “The split Bregman method for L1-regularized problems,” SIAM J. Imag. Sci., vol. 2, no. 2, pp. 323–343, 2009.
- M. Lustig, D. Donoho, and J. M. Pauly, “Sparse MRI: The application of compressed sensing for rapid MR imaging,” Magn. Reson. Med., vol. 58, no. 6, pp. 1182–1195, 2007.
- H. Jung, K. Sung, K. S. Nayak, E. Y. Kim, and J. C. Ye, “k-t FOCUSS: A general compressed sensing framework for high resolution dynamic MRI,” Magn. Reson. Med., vol. 61, no. 1, pp. 103–116, 2009.
- M. Guerquin-Kern, M. Häberlin, K. P. Pruessmann, and M. Unser, “A fast wavelet-based reconstruction method for magnetic resonance imaging,” IEEE Trans. Med. Imag., vol. 30, no. 9, pp. 1649–1660, Sep. 2011.
- E. Y. Sidky and X. Pan, “Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization,” Phys. Med. Biol., vol. 53, no. 17, p. 4777, Sep. 2008.
- M. G. McGaffin and J. A. Fessler, “Alternating dual updates algorithm for X-ray CT reconstruction on the GPU,” IEEE Trans. Comput. Imag., vol. 1, no. 3, pp. 186–199, Sep. 2015.
- M. T. McCann, M. Nilchian, M. Stampanoni, and M. Unser, “Fast 3D reconstruction method for differential phase contrast X-ray CT,” Opt. Exp., vol. 24, no. 13, pp. 14564–14581, Jun. 2016.
- Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, pp. 436–444, May 2015.
- A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1097–1105.
- O. Russakovsky et al., “ImageNet large scale visual recognition challenge,” Int. J. Comput. Vis., vol. 115, no. 3, pp. 211–252, Dec. 2015.
- R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 580–587.
- J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 3431–3440.
- O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Intervent, 2015, pp. 234–241.
- H. C. Burger, C. J. Schuler, and S. Harmeling, “Image denoising: Can plain neural networks compete with BM3D?” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp. 2392–2399.
- J. Xie, L. Xu, and E. Chen, “Image denoising and inpainting with deep neural networks,” in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 341–349.
- L. Xu, J. S. Ren, C. Liu, and J. Jia, “Deep convolutional neural network for image deconvolution,” in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 1790–1798.
- C. Dong, Y. Deng, C. Change Loy, and X. Tang, “Compression artifacts reduction by a deep convolutional network,” in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2015, pp. 576–584.
- J. Guo and H. Chao, “Building dual-domain representations for compression artifacts reduction,” in Proc. Eur. Conf. Comput. Vis., 2016, pp. 628–644.
- K. Kulkarni, S. Lohit, P. Turaga, R. Kerviche, and A. Ashok, “ReconNet: Non-iterative reconstruction of images from compressively sensed measurements,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jan. 2016, pp. 449–458.
- V. Golkov, A. Dosovitskiy, J. I. Sperl, M. I. Menzel, M. Czisch, P. Sämann, T. Brox, and D. Cremers, “q-space deep learning: Twelvefold shorter and model-free diffusion MRI scans,” IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1344–1351, May 2016.
- C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 2, pp. 295–307, Feb. 2015.
- J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 1646–1654.
- G. Riegler, M. Rüther, and H. Bischof, “ATGV-Net: Accurate depth super-resolution,” in Proc. Eur. Conf. Comput. Vis., 2016, pp. 268–284.
- K. Gregor and Y. LeCun, “Learning fast approximations of sparse coding,” in Proc. 27th Int. Conf. Mach. Learn. (ICML), 2010, pp. 399–406.
- Y. Chen, W. Yu, and T. Pock, “On learning optimized reaction diffusion processes for effective image restoration,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 5261–5269.
- B. Xin, Y. Wang, W. Gao, and D. Wipf, “Maximal sparsity with deep networks?” in Proc. Adv. Neural Inf. Process. Syst., 2016, pp. 4340–4348.
- A. Barbu, “Training an active random field for real-time image denoising,” IEEE Trans. Image Process., vol. 18, no. 11, pp. 2451–2462, Nov. 2009.
- U. Schmidt and S. Roth, “Shrinkage fields for effective image restoration,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 2774–2781.
- I. Daubechies, M. Defrise, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,” Commun. Pure Appl. Math., vol. 57, no. 11, pp. 1413–1457, Nov. 2004.
- U. S. Kamilov and H. Mansour, “Learning optimal nonlinearities for iterative thresholding algorithms,” IEEE Signal Process. Lett., vol. 23, no. 5, pp. 747–751, May 2016.
- K. P. Pruessmann, M. Weigner, M. B. Scheidegger, and P. Boesiger, “SENSE: Sensitivity encoding for fast MRI,” Magn. Reson. Med., vol. 42, no. 5, pp. 952–962, 1999.
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2016, pp. 770–778.
- A. Kak and M. Slaney, Principles of Computerized Tomographic Imaging. Philadelphia, PA, USA: Society Industrial Applied Mathematics, 2001.
- M. Beister, D. Kolditz, and W. A. Kalender, “Iterative reconstruction methods in X-ray CT,” Phys. Med., vol. 28, no. 2, pp. 94–108, Apr. 2012.
- S. Y. Chun, Y. K. Dewaraja, and J. A. Fessler, “Alternating direction method of multiplier for tomography with nonlocal regularizers,” IEEE Trans. Med. Imag., vol. 33, no. 10, pp. 1960–1968, Oct. 2014.
- Q. Xu, H. Yu, X. Mou, L. Zhang, J. Hsieh, and G. Wang, “Low-dose X-ray CT reconstruction via dictionary learning,” IEEE Trans. Med. Imaging, vol. 31, no. 9, pp. 1682–1697, Sep. 2012.
- D. M. Pelt and K. J. Batenburg, “Fast tomographic reconstruction from limited data using artificial neural networks,” IEEE Trans. Image Process., vol. 22, no. 12, pp. 5238–5251, Dec. 2013.
- D. Boublil, M. Elad, J. Shtok, and M. Zibulevsky, “Spatially-adaptive reconstruction in computed tomography using neural networks,” IEEE Trans. Med. Imag., vol. 34, no. 7, pp. 1474–1485, Jul. 2015.
- H. Chen, Y. Zhang, W. Zhang, P. Liao, K. Li, J. Zhou, and G. Wang, “Low-dose CT via convolutional neural network,” Biomed. Opt. Exp., vol. 8, no. 2, pp. 679–694, Feb. 2017.
- Y. Han, J. Yoo, and J. C. Ye. “Deep residual learning for compressed sensing ct reconstruction via persistent homology analysis.” [Online]. Available: https://arxiv.org/abs/1611.06391, 2016.
- S. Horbelt, M. Liebling, and M. Unser, “Discretization of the radon transform and of its inverse by spline convolutions,” IEEE Trans. Med. Imag., vol. 21, no. 4, pp. 363–376, Apr. 2002.
- R. Tibshirani, “Regression shrinkage and selection via the LASSO,” J. Roy. Statist Soc. Ser. B (Methodol.), vol. 58, no. 1, pp. 267–288, 1996.
- M. Elad, P. Milanfar, and R. Rubinstein, “Analysis versus synthesis in signal priors,” Inverse problems, vol. 23, no. 3, p. 947, 2007.
- A. L. da Cunha, J. Zhou, and M. N. Do, “The nonsubsampled contourlet transform: Theory, design, and applications,” IEEE Trans. Image Process., vol. 15, no. 10, pp. 3089–3101, Oct. 2006.
- S. Mallat, A Wavelet Tour of Signal Processing. San Diego, CA, USA: Academic, 1999.
- R. R. Coifman and D. L. Donoho, Translation-Invariant De-Noising. New York, NY, USA: Springer, 1995.
- S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learn., vol. 3, no. 1, pp. 1–122, Jan. 2011.
- A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM J. Imag. Sci., vol. 2, no. 1, pp. 183–202, 2009.
- M. V. Afonso, J.-M. Bioucas-Dias, and M. A. T. Figueiredo, “Fast image recovery using variable splitting and constrained optimization,” IEEE Trans. Image Process., vol. 19, no. 9, pp. 2345–2356, Sep. 2010.
- M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” in Proc. Eur. Conf. Comput. Vis., 2014, pp. 818–833.
- J. Cong and B. Xiao, “Minimizing computation in convolutional neural networks,” in Proc. Int. Conf. Artif. Neural Netw., 2014, pp. 281–290.
- M. Rhu, N. Gimelshein, J. Clemons, A. Zulfiqar, and S. W. Keckler, “vDNN: Virtualized deep neural networks for scalable, memory-efficient neural network design,” in Proc. 49th Annu. IEEE/ACM Int. Symp. Microarchitecture (MICRO), Oct. 2016, pp. 1–13.
- A. Vedaldi and K. Lenc, “Matconvnet: Convolutional neural networks for matlab,” in Proc. ACM Int. Conf. Multimedia, 2015, pp. 689–692.
- R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty of training recurrent neural networks,” in Proc. ICML, vol. 28. 2013, pp. 1310–1318.
- M. Unser, J. Fageot, and H. Gupta, “Representer theorems for sparsitypromoting 1 regularization,” IEEE Trans. Inf. Theory, vol. 62, no. 9, pp. 5167–5180, Sep. 2016.
- K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a Gaussian Denoiser: Residual learning of deep CNN for image denoising,” IEEE Trans. Image Process., vol. 26, no. 7, pp. 3142–3155, Jul. 2017.
- H. Lee, J. Lee, and S. Cho, “View-interpolation of sparsely sampled sinogram using convolutional neural network,” Proc. SPIE, vol. 10133, p. 1013328, Feb. 2017.
- K. Epperson, A. M. Sawyer, M. Lustig, M. Alley, M. Uecker, P. Virtue, P. Lei, and S. Vasanawala, “Creation of fully sampled mr data repository for compressed sensing of the knee,” in Proc. SMRT Conf., 2013. Kyong Hwan Jin received the B.S. and the integrated M.S. and Ph.D. degrees from the Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea, in 2008 and 2015, respectively. He was a Post-Doctoral Scholar with KAIST from 2015 to 2016. He is currently a Post-Doctoral Scholar with the Biomedical Imaging Group, École Polytechnique Fédérale de Lausanne, Switzerland. His research interests include low rank matrix completion, sparsity promoted signal recovery, sampling theory, biomedical imaging, and image processing in various applications.
- Michael T. McCann (S’10–M’15) received the B.S.E. in biomedical engineering from the University of Michigan in 2010 and the Ph.D. degree in biomedical engineering from Carnegie Mellon University in 2015. He is currently a Scientist with the Laboratoire d’Imagerie biomédicale and centre d’imagerie biomédicale, École Polytechnique Fédérale de Lausanne, where he is involved in X-ray CT reconstruction. His research interest centers on developing signal processing tools to answer biomedical questions.
- Emmanuel Froustey received the Dipl.Eng. degree from CentraleSupélec, Châtenay-Malabry, France, in 2012, and the M.Sc. degree in computational science and engineering from the École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, in 2014. He is currently a Research and Development Engineer with Dassault Aviation. His research interests include variational models and optimization algorithms with applications to biomedical imaging.