SentiGAN: Generating Sentimental Texts via Mixture Adversarial Networks

IJCAI, pp. 4446-4452, 2018.

Cited by: 55|Bibtex|Views164|Links
EI
Keywords:
Customer Reviewssupervised VAEdeep generative modelVariational AutoencodersMovie ReviewsMore(15+)
Weibo:
We propose SentiGAN, which can generate a variety of high-quality texts of different sentiment labels

Abstract:

Generating texts of different sentiment labels is getting more and more attention in the area of natural language generation. Recently, Generative Adversarial Net (GAN) has shown promising results in text generation. However, the texts generated by GAN usually suffer from the problems of poor quality, lack of diversity and mode collapse. ...More

Code:

Data:

0
Introduction
  • Emotional intelligence is an important part of artificial intelligence. Automatic understanding and generation of sentimental texts make machines more friendly to humans, and make them look more intelligent.
  • Previous work has been mostly limited to task-specific applications and just use hidden variables to indirectly control the sentiment labels of generated texts, especially in emotional response generation [Zhou et al, 2017; Zhou and Wang, 2017].
  • Generative Adversarial Net (GAN) [Goodfellow et al, 2014] is a good solution to this problem which uses a discriminator instead of a specific objective to guide the generator.
  • The main intuition is that since text sentiment classification is very strong, the authors can use the classifier to guide the generation of sentimental texts
Highlights
  • Emotional intelligence is an important part of artificial intelligence
  • 2) We propose a new penalty based objective to make each generator in SentiGAN produce diversified texts of a specific sentiment label
  • The average values over generated sentences are shown in Table 2, we can see that RNNLM, SeqGAN and Variational Autoencoders are not good at generating new sentences
  • 4.5 Case Study In Table 4, we show example sentences generated by SentiGAN(k=2) and C-Generative Adversarial Net trained on the Movie Reviews dataset
  • We propose SentiGAN, which can generate a variety of high-quality texts of different sentiment labels
  • Extensive experiments demonstrate the efficacy of SentiGAN
Methods
  • The average values over generated sentences are shown in Table 2, the authors can see that RNNLM, SeqGAN and VAE are not good at generating new sentences.
  • The average values are shown in Table 3, and the authors can see that the model can generate a variety of sentences, while other models can not ensure the diversity of generated sentences.
  • The authors can see that the model performs better than all other methods and the model can generate sentimental sentences with best intelligibility.
  • Comparing the results on different datasets, the authors can see that more data can train better models with respect to intelligibility (CR < M R < BR)
Results
  • Experimental results on four datasets demonstrate that the model consistently outperforms several state-of-the-art text generation methods in the sentiment accuracy and quality of generated texts.
Conclusion
  • The authors propose SentiGAN, which can generate a variety of high-quality texts of different sentiment labels.
  • Extensive experiments demonstrate the efficacy of SentiGAN.
  • The authors will make use of more complex and sophisticated generators to enhance the quality of generated texts, especially for longer text generation.
  • The authors will apply the model to generate texts with other kinds of labels
Summary
  • Introduction:

    Emotional intelligence is an important part of artificial intelligence. Automatic understanding and generation of sentimental texts make machines more friendly to humans, and make them look more intelligent.
  • Previous work has been mostly limited to task-specific applications and just use hidden variables to indirectly control the sentiment labels of generated texts, especially in emotional response generation [Zhou et al, 2017; Zhou and Wang, 2017].
  • Generative Adversarial Net (GAN) [Goodfellow et al, 2014] is a good solution to this problem which uses a discriminator instead of a specific objective to guide the generator.
  • The main intuition is that since text sentiment classification is very strong, the authors can use the classifier to guide the generation of sentimental texts
  • Methods:

    The average values over generated sentences are shown in Table 2, the authors can see that RNNLM, SeqGAN and VAE are not good at generating new sentences.
  • The average values are shown in Table 3, and the authors can see that the model can generate a variety of sentences, while other models can not ensure the diversity of generated sentences.
  • The authors can see that the model performs better than all other methods and the model can generate sentimental sentences with best intelligibility.
  • Comparing the results on different datasets, the authors can see that more data can train better models with respect to intelligibility (CR < M R < BR)
  • Results:

    Experimental results on four datasets demonstrate that the model consistently outperforms several state-of-the-art text generation methods in the sentiment accuracy and quality of generated texts.
  • Conclusion:

    The authors propose SentiGAN, which can generate a variety of high-quality texts of different sentiment labels.
  • Extensive experiments demonstrate the efficacy of SentiGAN.
  • The authors will make use of more complex and sophisticated generators to enhance the quality of generated texts, especially for longer text generation.
  • The authors will apply the model to generate texts with other kinds of labels
Tables
  • Table1: Comparison of sentiment accuracy of generated sentences. The real data is the training corpus
  • Table2: Comparison of the novelty of generated sentences
  • Table3: Comparison of the diversity of generated sentences
  • Table4: Examples sentences generated by SentiGAN and Conditional GAN trained on MR
  • Table5: The performance comparison of different methods on the synthetic data in terms of the negative log-likelihood (NLL) scores
Download tables as Excel
Related work
  • Unsupervised text generation is an important research area in natural language processing. A standard recurrent neural network language model [Mikolov et al, 2011] predicts each word of a sentence conditioned on the previous word and an evolving hidden state. However, it suffers from two major drawbacks when used to generate texts. First, RNN based models are always trained through the maximum likelihood approach, which suffers from exposure bias [Bengio et al, 2015]. Second, the loss function used to train the model is at word level but the performance is typically evaluated at sentence level. There are some researches which use generative adversarial network (GAN) to solve the problems.
Funding
  • This work was supported by National Natural Science Foundation of China (61772036, 61331011) and Key Laboratory of Science, Technology and Standard in Press Industry (Key Laboratory of Intelligent Press Media Technology)
Reference
  • [Arjovsky et al., 2017] Martın Arjovsky, Soumith Chintala, and Leon Bottou. Wasserstein GAN. CoRR, abs/1701.07875, 2017.
    Findings
  • [Bachman and Precup, 2015] Philip Bachman and Doina Precup. Data generation as sequential decision making. NIPS, 2015.
    Google ScholarLocate open access versionFindings
  • [Bengio et al., 2015] Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. In NIPS, pages 1171–1179, 2015.
    Google ScholarLocate open access versionFindings
  • [Denton et al., 2015] Emily L Denton, Soumith Chintala, Arthur Szlam, and Robert D Fergus. Deep generative image models using a laplacian pyramid of adversarial networks. NIPS, pages 1486–1494, 2015.
    Google ScholarLocate open access versionFindings
  • [Dong et al., 2017] Li Dong, Shaohan Huang, Furu Wei, Mirella Lapata, Ming Zhou, and Ke Xu. Learning to generate product reviews from attributes. In EACL, volume 1, pages 623–632, 2017.
    Google ScholarLocate open access versionFindings
  • [Goodfellow et al., 2014] Ian J Goodfellow, Jean Pougetabadie, Mehdi Mirza, Bing Xu, David Wardefarley, Sherjil Ozair, Aaron C Courville, and Yoshua Bengio. Generative adversarial networks. arXiv: Machine Learning, 2014.
    Google ScholarFindings
  • [Guo et al., 2017] Jiaxian Guo, Sidi Lu, Han Cai, Weinan Zhang, Yong Yu, and Jun Wang. Long text generation via adversarial training with leaked information. CoRR, 2017.
    Google ScholarFindings
  • [Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jurgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997.
    Google ScholarLocate open access versionFindings
  • [Hu and Liu, 2004] Minqing Hu and Bing Liu. Mining and summarizing customer reviews. pages 168–177, 2004.
    Google ScholarFindings
  • [Hu et al., 2016] Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard H Hovy, and Eric P Xing. Harnessing deep neural networks with logic rules. ACL, pages 2410–2420, 2016.
    Google ScholarLocate open access versionFindings
  • [Hu et al., 2017] Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. Toward controlled generation of text. In ICML, pages 1587–1596, 2017.
    Google ScholarLocate open access versionFindings
  • [Isola et al., 2016] Phillip Isola, Junyan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. CVPR, 2016.
    Google ScholarLocate open access versionFindings
  • [Kingma and Welling, 2014] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. ICLR, 2014.
    Google ScholarLocate open access versionFindings
  • [Kingma et al., 2014] Diederik P Kingma, Danilo J Rezende, Shakir Mohamed, and Max Welling. Semisupervised learning with deep generative models. NIPS, 4:3581–3589, 2014.
    Google ScholarLocate open access versionFindings
  • [Kusner and Hernandez-Lobato, 2016] Matt J. Kusner and Jose Miguel Hernandez-Lobato. GANS for sequences of discrete elements with the gumbel-softmax distribution. CoRR, abs/1611.04051, 2016.
    Findings
  • [Lamb et al., 2016] Alex Lamb, Anirudh Goyal, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. Professor forcing: A new algorithm for training recurrent networks. NIPS, pages 4601–4609, 2016.
    Google ScholarLocate open access versionFindings
  • [Lin et al., 2017] Kevin Lin, Dianqi Li, Xiaodong He, MingTing Sun, and Zhengyou Zhang. Adversarial ranking for language generation. In NIPS, pages 3158–3168, 2017.
    Google ScholarLocate open access versionFindings
  • [Lipton et al., 2015] Zachary C Lipton, Sharad Vikram, and Julian McAuley. Generative concatenative nets jointly learn to write and classify reviews. arXiv preprint arXiv:1511.03683, 2015.
    Findings
  • [Mcauley and Leskovec, 2013] Julian Mcauley and Jure Leskovec. From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews. WWW, pages 897–908, 2013.
    Google ScholarLocate open access versionFindings
  • [Mikolov et al., 2011] Tomas Mikolov, Stefan Kombrink, Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur. Extensions of recurrent neural network language model. pages 5528–5531, 2011.
    Google ScholarFindings
  • [Mirza and Osindero, 2014] Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv: Learning, 2014.
    Google ScholarFindings
  • [Salimans et al., 2016] Tim Salimans, Ian J Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. NIPS, pages 2234–2242, 2016.
    Google ScholarLocate open access versionFindings
  • [Socher et al., 2013] Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. 2013.
    Google ScholarFindings
  • [Stolcke, 2002] Andreas Stolcke. Srilm – an extensible language modeling toolkit. 2002.
    Google ScholarFindings
  • [Theis et al., 2016] Lucas Theis, Aaron Van Den Oord, and Matthias Bethge. A note on the evaluation of generative models. ICLR, 2016.
    Google ScholarLocate open access versionFindings
  • [Yu et al., 2017] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In AAAI, 2017.
    Google ScholarFindings
  • [Zang and Wan, 2017] Hongyu Zang and Xiaojun Wan. Towards automatic generation of product reviews from aspect-sentiment scores. In INLG, pages 168–177, 2017.
    Google ScholarLocate open access versionFindings
  • [Zhang and Lecun, 2015] Xiang Zhang and Yann Lecun. Text understanding from scratch. arXiv: Learning, 2015.
    Google ScholarFindings
  • [Zhou and Wang, 2017] Xianda Zhou and William Yang Wang. Mojitalk: Generating emotional responses at scale. CoRR, abs/1711.04090, 2017.
    Findings
  • [Zhou et al., 2017] Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. Emotional chatting machine: Emotional conversation generation with internal and external memory. CoRR, abs/1704.01074, 2017.
    Findings
Your rating :
0

 

Best Paper
Best Paper of IJCAI, 2018
Tags
Comments