A Scalable System to Synthesize Data for Natural Scene Text Localization and Recognition

In this paper, a scalable system is proposed to synthesize data for multilingual text (Chinese and English) detection and recognition in cluttered conditions. The system can be realized by these following steps. First, in order to respect scene boundaries when overlaying synthetic text to existing images, hierarchical contour is obtained by using efficient graph-segmentation. Second, in order to paint text on top of surfaces with approximate natural effect, scene 3D geometry is estimated under a single image by using Convolutional Neural Network (CNN) based processing. Finally, in order to synthesize data more naturally, the text is printed into the picture appropriately by poisson image editing and the text color is rendered by a combination of foreground and background colors. The system can generate scene text data automatically and naturally, providing a cheap way to annotate massive images to train supervisor learning model for natural scene text detection and recognition.