Unsupervised Learning of Intrinsic Structural Representation Points

CVPR, pp. 9118-9127, 2020.

Cited by: 0|Bibtex|Views43|Links
EI
Keywords:
Chamfer distance3d shapestructural representationunsupervised learningnew structural representationMore(13+)
Weibo:
We present a simple yet interpretable unsupervised method for learning a new structural representation in the form of 3D structure points

Abstract:

Learning structures of 3D shapes is a fundamental problem in the field of computer graphics and geometry processing. We present a simple yet interpretable unsupervised method for learning a new structural representation in the form of 3D structure points. The 3D structure points pro-duced by our method encode the shape structure intrins...More

Code:

Data:

0
Introduction
  • Analyzing 3D shape structures is a fundamental problem in the field of computer graphics and geometry processing.
  • Quite a few structural representations have been proposed for 3D shapes, such as medial axis, curve skeleton and keypoints, which are designed for different tasks.
  • Works mainly use hand-crafted features and formulate it as optimization problems.
  • While, they usually rely on parameter tuning, and are designed for specific tasks or datasets.
  • The authors propose a method to learn a new structural representation for establishing semantic correspondence for 3D point clouds
Highlights
  • Analyzing 3D shape structures is a fundamental problem in the field of computer graphics and geometry processing
  • Quite a few structural representations have been proposed for 3D shapes, such as medial axis, curve skeleton and keypoints, which are designed for different tasks
  • We propose a method to learn a new structural representation for establishing semantic correspondence for 3D point clouds
  • Our PCA based structure points embedding has the potential to be used in some important tasks like shape reconstruction and completion
  • We present a simple yet interpretable unsupervised method for learning a new structural representation in the form of 3D structure points
Methods
  • The authors introduce the end-to-end framework for learning intrinsic structure points from point clouds without explicit supervision.
  • The network is trained for a collection of 3D shapes in the same category in an unsupervised manner.
  • The authors first describe the network architecture which is composed of a PointNet++ encoder and a point integration module.
  • The authors show that the produced structure points exhibit semantic consistency across all the shapes in the same category, which is an essential property for shape co-analysis
Results
  • Extensive experiments have shown that the method outperforms the state-of-the-art on the semantic shape correspondence task and achieves comparable performance with the state-of-the-art on the segmentation label transfer task.
  • The authors' method achieves state-of-the-art performance on the semantic shape correspondence task and the segmentation label transfer task.
  • The authors' method outperforms the state-of-the-art on the semantic shape correspondence task and achieves comparable performance with state-of-the-art methods on the segmentation label transfer task
Conclusion
  • The authors present a simple yet interpretable unsupervised method for learning a new structural representation in the form of 3D structure points.
  • The produced structure points encode shape structures and exhibit semantic consistency across all the shape instances with similar shape structures.
  • The authors evaluate the proposed method with extensive experiments and show state-of-the-art performance on both semantic correspondence and segmentation label transfer tasks.
  • The authors' PCA based structure points embedding has the potential to be used in some important tasks like shape reconstruction and completion, which the authors would like to explore more in the future
Summary
  • Introduction:

    Analyzing 3D shape structures is a fundamental problem in the field of computer graphics and geometry processing.
  • Quite a few structural representations have been proposed for 3D shapes, such as medial axis, curve skeleton and keypoints, which are designed for different tasks.
  • Works mainly use hand-crafted features and formulate it as optimization problems.
  • While, they usually rely on parameter tuning, and are designed for specific tasks or datasets.
  • The authors propose a method to learn a new structural representation for establishing semantic correspondence for 3D point clouds
  • Methods:

    The authors introduce the end-to-end framework for learning intrinsic structure points from point clouds without explicit supervision.
  • The network is trained for a collection of 3D shapes in the same category in an unsupervised manner.
  • The authors first describe the network architecture which is composed of a PointNet++ encoder and a point integration module.
  • The authors show that the produced structure points exhibit semantic consistency across all the shapes in the same category, which is an essential property for shape co-analysis
  • Results:

    Extensive experiments have shown that the method outperforms the state-of-the-art on the semantic shape correspondence task and achieves comparable performance with the state-of-the-art on the segmentation label transfer task.
  • The authors' method achieves state-of-the-art performance on the semantic shape correspondence task and the segmentation label transfer task.
  • The authors' method outperforms the state-of-the-art on the semantic shape correspondence task and achieves comparable performance with state-of-the-art methods on the segmentation label transfer task
  • Conclusion:

    The authors present a simple yet interpretable unsupervised method for learning a new structural representation in the form of 3D structure points.
  • The produced structure points encode shape structures and exhibit semantic consistency across all the shape instances with similar shape structures.
  • The authors evaluate the proposed method with extensive experiments and show state-of-the-art performance on both semantic correspondence and segmentation label transfer tasks.
  • The authors' PCA based structure points embedding has the potential to be used in some important tasks like shape reconstruction and completion, which the authors would like to explore more in the future
Tables
  • Table1: Comparison of our label transfer results against BAE-NET with 3 labeled exemplars on the ShapeNet part dataset[<a class="ref-link" id="c43" href="#r43">43</a>] measured with average IOU(%)
  • Table2: Stability of the network for producing 1024 structure points with input points of different uniform densities
Download tables as Excel
Related work
  • 2.1. Shape Structure Analysis

    Recently, quite a few works have been proposed for learning keypoints as structural shape representations. Several unsupervised methods have been proposed to learn keypoints in 2D image domain. [16, 24, 46] disentangle the structure and appearance of 2D images for keypoints discovery. For keypoint detection on 3D shapes, KeyPointNet [35] utilizes multi-view consistency to discover a sparse set of geometrically and semantically consistent keypoints across different shapes in the same category.

    Unlike previous works, we conduct unsupervised learning of either sparse or dense consistent structure points as structural representation directly on 3D point clouds. Our method can be easily generalized to real scanned data.
Funding
  • Presents a simple yet interpretable unsupervised method for learning a new structural representation in the form of 3D structure points
  • Extensive experiments have shown that our method outperforms the state-of-the-art on the semantic shape correspondence task and achieves comparable performance with the state-of-the-art on the segmentation label transfer task
  • Proposes a method to learn a new structural representation for establishing semantic correspondence for 3D point clouds
  • Our method outperforms the state-of-the-art on the semantic shape correspondence task and achieves comparable performance with state-of-the-art methods on the segmentation label transfer task
Reference
  • Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning representations and generative models for 3d point clouds. arXiv preprint arXiv:1707.02392, 2017. 2, 5
    Findings
  • Volker Blanz, Thomas Vetter, et al. A morphable model for the synthesis of 3d faces. In Siggraph, volume 99, pages 187–194, 1999. 6
    Google ScholarLocate open access versionFindings
  • Mingjia Chen, Qianfang Zou, Changbo Wang, and Ligang Liu. Edgenet: Deep metric learning for 3d shapes. CAGD, 72:19–33, 2019. 2, 5
    Google ScholarLocate open access versionFindings
  • Zhiqin Chen, Kangxue Yin, Matthew Fisher, Siddhartha Chaudhuri, and Hao Zhang. Bae-net: Branched autoencoder for shape co-segmentation. arXiv preprint arXiv:1903.11228, 2019. 1, 5, 6
    Findings
  • Christopher Choy, Jaesik Park, and Vladlen Koltun. Fully convolutional geometric features. In ICCV, 2019. 2
    Google ScholarLocate open access versionFindings
  • Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017. 8
    Google ScholarLocate open access versionFindings
  • Haowen Deng, Tolga Birdal, and Slobodan Ilic. Ppf-foldnet: Unsupervised learning of rotation invariant 3d local descriptors. In ECCV, pages 602–618, 2018. 2
    Google ScholarLocate open access versionFindings
  • Theo Deprelle, Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. Learning elementary structures for 3d shape generation and matching. arXiv preprint arXiv:1908.04725, 2019. 1, 2, 5
    Findings
  • Yuval Eldar, Michael Lindenbaum, Moshe Porat, and Yehoshua Y Zeevi. The farthest point strategy for progressive image sampling. TIP, 6(9):1305–1315, 1997. 4
    Google ScholarLocate open access versionFindings
  • Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object reconstruction from a single image. In CVPR, pages 605–613, 2017. 2, 4
    Google ScholarLocate open access versionFindings
  • Zan Gojcic, Caifa Zhou, Jan D Wegner, and Andreas Wieser. The perfect match: 3d point cloud matching with smoothed densities. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5545–5554, 2019. 2
    Google ScholarLocate open access versionFindings
  • Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. Atlasnet: A papierm\ˆ ach\’e approach to learning 3d surface generation. arXiv preprint arXiv:1802.05384, 2018. 2
    Findings
  • Haibin Huang, Evangelos Kalogerakis, Siddhartha Chaudhuri, Duygu Ceylan, Vladimir G Kim, and Ersin Yumer. Learning local shape descriptors from part correspondences with multiview convolutional networks. TOG, 37(1):6, 2018. 1, 2, 4, 5
    Google ScholarLocate open access versionFindings
  • Qixing Huang, Fan Wang, and Leonidas Guibas. Functional map networks for analyzing and exploring large shape collections. TOG, 33(4):36, 201, 2
    Google ScholarLocate open access versionFindings
  • Qi-Xing Huang, Hao Su, and Leonidas Guibas. Fine-grained semi-supervised labeling of large shape collections. TOG, 32(6):190, 2013. 2
    Google ScholarLocate open access versionFindings
  • Tomas Jakab, Ankush Gupta, Hakan Bilen, and Andrea Vedaldi. Conditional image generation for learning the structure of visual objects. methods, 43:44, 2018. 2
    Google ScholarLocate open access versionFindings
  • Ian Jolliffe. Principal component analysis. Springer, 2011. 5
    Google ScholarFindings
  • Vladimir G Kim, Wilmot Li, Niloy J Mitra, Siddhartha Chaudhuri, Stephen DiVerdi, and Thomas Funkhouser. Learning part-based templates from large collections of 3d shapes. TOG, 32(4):70, 2013. 2, 4, 5
    Google ScholarLocate open access versionFindings
  • Vladimir G Kim, Wilmot Li, Niloy J Mitra, Stephen DiVerdi, and Thomas A Funkhouser. Exploring collections of 3d models using fuzzy correspondences. TOG, 31(4):54–1, 2012. 2
    Google ScholarLocate open access versionFindings
  • Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. 4
    Findings
  • Jiaxin Li, Ben M Chen, and Gim Hee Lee. So-net: Selforganizing network for point cloud analysis. In CVPR, pages 9397–9406, 2018. 2
    Google ScholarLocate open access versionFindings
  • Lingxiao Li, Minhyuk Sung, Anastasia Dubrovina, Li Yi, and Leonidas J Guibas. Supervised fitting of geometric primitives to 3d point clouds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2652–2660, 2019. 1
    Google ScholarLocate open access versionFindings
  • Chen-Hsuan Lin, Chen Kong, and Simon Lucey. Learning efficient point cloud generation for dense 3d object reconstruction. In AAAI, 2018. 2
    Google ScholarLocate open access versionFindings
  • Dominik Lorenz, Leonard Bereska, Timo Milbich, and Bjorn Ommer. Unsupervised part-based disentangling of object shape and appearance. arXiv preprint arXiv:1903.06946, 2019. 2
    Findings
  • Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579–2605, 2008. 7
    Google ScholarLocate open access versionFindings
  • Sanjeev Muralikrishnan, Vladimir G Kim, Matthew Fisher, and Siddhartha Chaudhuri. Shape unicode: A unified shape representation. In CVPR, pages 3790–3799, 2019. 1, 2, 5
    Google ScholarLocate open access versionFindings
  • Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. Functional maps: a flexible representation of maps between shapes. TOG, 31(4):30, 2012. 2
    Google ScholarLocate open access versionFindings
  • Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, pages 652–660, 2017. 1, 2, 5
    Google ScholarLocate open access versionFindings
  • Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Neurips, pages 5099–5108, 2017. 2, 3, 8
    Google ScholarLocate open access versionFindings
  • Emanuele Rodola, Samuel Rota Bulo, Thomas Windheuser, Matthias Vestner, and Daniel Cremers. Dense non-rigid shape correspondence using random forests. In CVPR, pages 4177–4184, 2014. 2
    Google ScholarLocate open access versionFindings
  • Dong Wook Shu, Sung Woo Park, and Junseok Kwon. 3d point cloud generative adversarial network based on tree structured graph convolutions. arXiv preprint arXiv:1905.06292, 2019. 1, 2
    Findings
  • Martin Simonovsky and Nikos Komodakis. Dynamic edgeconditioned filters in convolutional neural networks on graphs. In CVPR, pages 3693–3702, 2017. 2
    Google ScholarLocate open access versionFindings
  • Chunyu Sun, Qianfang Zou, Xin Tong, and Yang Liu. Learning adaptive hierarchical cuboid abstractions of 3d shape collections. ACM Transactions on Graphics (SIGGRAPH Asia), 38(6), 2019. 1
    Google ScholarLocate open access versionFindings
  • Minhyuk Sung, Hao Su, Ronald Yu, and Leonidas J Guibas. Deep functional dictionaries: Learning consistent semantic structures on 3d models from functions. In Advances in Neural Information Processing Systems, pages 485–495, 2018. 2
    Google ScholarLocate open access versionFindings
  • Supasorn Suwajanakorn, Noah Snavely, Jonathan J Tompson, and Mohammad Norouzi. Discovery of latent 3d keypoints via end-to-end geometric reasoning. In Neurips, pages 2059–2070, 2018. 1, 2
    Google ScholarLocate open access versionFindings
  • Lyne P Tchapmi, Vineet Kosaraju, Hamid Rezatofighi, Ian Reid, and Silvio Savarese. Topnet: Structural point cloud decoder. In CVPR, pages 383–392, 2019. 1, 2, 5
    Google ScholarLocate open access versionFindings
  • Shubham Tulsiani, Hao Su, Leonidas J Guibas, Alexei A Efros, and Jitendra Malik. Learning shape abstractions by assembling volumetric primitives. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2635–2643, 2017. 1
    Google ScholarLocate open access versionFindings
  • Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. arXiv preprint arXiv:1801.07829, 2018. 2
    Findings
  • Wenxuan Wu, Zhongang Qi, and Li Fuxin. Pointconv: Deep convolutional networks on 3d point clouds. In CVPR, pages 9621–9630, 2019. 2, 8
    Google ScholarLocate open access versionFindings
  • Kai Xu, Vladimir G Kim, Qixing Huang, Niloy Mitra, and Evangelos Kalogerakis. Data-driven shape analysis and processing. In SIGGRAPH ASIA, page 4. ACM, 2016. 2
    Google ScholarLocate open access versionFindings
  • Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE International Conference on Computer Vision, pages 4541–4550, 2019. 1, 2
    Google ScholarLocate open access versionFindings
  • Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In CVPR, pages 206–215, 2018. 2, 5
    Google ScholarLocate open access versionFindings
  • Li Yi, Vladimir G Kim, Duygu Ceylan, I Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, Leonidas Guibas, et al. A scalable active framework for region annotation in 3d shape collections. TOG, 35(6):210, 2016. 4, 5, 7
    Google ScholarLocate open access versionFindings
  • Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. Pu-net: Point cloud upsampling network. In CVPR, pages 2790–2799, 2018. 2
    Google ScholarLocate open access versionFindings
  • Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In CVPR, pages 1802–1811, 2017. 2
    Google ScholarLocate open access versionFindings
  • Yuting Zhang, Yijie Guo, Yixin Jin, Yijun Luo, Zhiyuan He, and Honglak Lee. Unsupervised discovery of object landmarks as structural representations. In CVPR, pages 2694– 2703, 2018. 2
    Google ScholarLocate open access versionFindings
  • Yongheng Zhao, Tolga Birdal, Haowen Deng, and Federico Tombari. 3d point capsule networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1009–1018, 2019. 1, 2, 5
    Google ScholarLocate open access versionFindings
  • Chenyang Zhu, Kai Xu, Siddhartha Chaudhuri, Li Yi, Leonidas Guibas, and Hao Zhang. Cosegnet: Deep cosegmentation of 3d shapes with group consistency loss. arXiv preprint arXiv:1903.10297, 2019. 1
    Findings
Your rating :
0

 

Tags
Comments