3D object classification and retrieval with Spherical CNNs.

3D object classification and retrieval presents many challenges that are not present in the traditional (planar) image setting. First, there is the question of shape representation. Face-vertex meshes, for instance, are widely used in computer graphics, but their irregular structure complicate their use as inputs to learning models. Previous works have converted meshes to more structured representations, such a collections of rendered views or volumetric grids, in order to feed them to 2D or 3D CNNs. These representations, however, are redundant and wasteful, requiring large amounts of storage, pre-processing time, and large networks with millions of parameters to handle them. Another challenge is how to treat object orientations. Orientation-invariance is a desired property for any classification engine, yet most current models do not address this explicitly, rather requiring increased model and sample complexity to handle arbitrary input orientations. We present a model that aims to be efficient in both the number of learnable parameters and input size. We leverage the group convolution equivariance properties; more specifically, the spherical convolution, to build a network that learns feature maps equivariant to SO(3) actions by design. By mapping a 3D input to the surface of a sphere, we also end up with a small input size.