Adjustable method based on body parts for improving the accuracy of 3D reconstruction in visually important body parts from silhouettes

This research proposes a novel adjustable algorithm for reconstructing 3D body shapes from front and side silhouettes. Most recent silhouette-based approaches use a deep neural network trained by silhouettes and key points to estimate the shape parameters but cannot accurately fit the model to the body contours and consequently are struggling to cover detailed body geometry, especially in the torso. In addition, in most of these cases, body parts have the same accuracy priority, making the optimization harder and avoiding reaching the optimum possible result in essential body parts, like the torso, which is visually important in most applications, such as virtual garment fitting. In the proposed method, the expected accuracy for each body part can be adjusted based on the purpose by assigning coefficients for the distance of each body part between the projected 3D body and 2D silhouettes. To measure this distance, the correspondent body parts are first recognized using body segmentation in both views. Individual body parts are aligned by 2D rigid registration and matched using pairwise matching. The objective function tries to minimize the distance cost for the individual body parts in both views based on distances and coefficients by optimizing the statistical model parameters. The slight variation in the degree of arms and limbs is also handled by matching the pose. The proposed method was evaluated with images of synthetic and real body meshes. The result shows that the algorithm accurately reconstructs visually important body parts with high coefficients.