Style creation: multiple styles transfer with incremental learning and distillation loss

Neural style transfer aims to transfer style from a style image to a content image by neural learning. A novel style will be brought if one can transfer more than one style to the content image. Because multiple style images can provide different textures and colors for style transfer. In this work, our goal is to train an image generator network to transfer multiple styles to a content image for users to create the satisfactory novel style. Ideally, given a content image and several style images, it is favorable that users can choose different style images according to personal preference to create a novel style because aesthetic preference may differ from user to user. In addition, we found that it is difficult for users to determine which style images can be used to generate a satisfactory novel style. To tackle these challenges, we formulate neural style transfer as an incremental learning process which is essentially trying to make the generator network remember old styles when learning a new style. A perceptual loss is used to train a feed-forward network to learn a new style, while the distillation loss which contain a style difference loss and a texture enhancement loss is designed to remember old styles during training of the network. Experiments demonstrate that during this process, users can drive the stylized result closer to the satisfactory novel style.