Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes

Most learning methods for 3D data suffer significant performance drops when the data is not carefully aligned to a canonical orientation. Aligning real world 3D data collected from different sources is non-trivial and requires manual intervention. In this paper, we propose the Adjoint Rigid Transform (ART) Network, a neural module which can be integrated with a variety of 3D networks to significantly boost their performance. ART learns to rotate input shapes to a learned canonical orientation, which is crucial for a lot of tasks such as shape reconstruction, interpolation, non-rigid registration, and latent disentanglement. ART achieves this with self-supervision and a rotation equivariance constraint on predicted rotations. With only self-supervision, ART facilitates learning a canonical orientation for both rigid and nonrigid shapes, which leads to a notable boost in performance of aforementioned tasks. Our code and model are available at [2].