Deep Sketch-Based Modeling of Man-Made Shapes.

Sketch-based modeling aims to model 3D geometry using a concise and easy to create---but extremely ambiguous---input: artist sketches. Most conventional sketch-based modeling systems target smooth shapes and, to counter the ambiguity, put manually-designed priors on the 3D shape; they also typically require clean, vectorized input. Recent approaches attempt to learn those priors from data but often produce low-quality output. Focusing on piecewise-smooth man-made shapes, we address these issues by presenting a deep learning-based system to infer a complete man-made 3D shape from a single bitmap sketch. Given a sketch, our system infers a set of parametric surfaces that realize the drawing in 3D. To capture the piecewise smooth geometry of man-made shapes, we learn a special shape representation---a deformable parametric template composed of Coons patches. Naively training such a system, however, would suffer from lack of data and from self-intersections of the parametric surfaces. To address this, we introduce a synthetic sketch augmentation pipeline as well as a loss function that biases the network to output non-self-intersecting shapes. We demonstrate the efficacy of our system on a gallery of synthetic and real artist sketches as well as via comparison to related work.