Weakly Supervised Joint Transfer and Regression of Textures for 3D Human Reconstruction

3D human model reconstruction from a single image has a wide range of applications in VR/AR, video games, virtual dressing and data synthesis. Texture generation is one of its important components. Existing texture flow based generation methods usually cannot handle well texture inference for invisible human parts in input images, especially for asymmetric texture structures. To address this problem, we propose a joint texture transfer and regression model consisting of part-aware texture transfer, which predicts the texture flow in the UV space from the semantic mesh segmentation, and UV-map based texture regression, which further refines textures for invisible human parts according to the UV texture map produced by the texture transfer. These two modules are learned jointly in a weakly supervised manner based on the perceptual metrics, without using 3D texture supervision. Furthermore, we exploit our generated textures to synthesize human images with various viewpoints, poses and backgrounds as training images for the person re-identification (re-ID) task. Extensive experiments on pedestrian image datasets show that our model produces higher quality textures than other texture generation methods and improves the re-ID performance.