Chinese Poetry Generation with Planning based Neural Network

COLING, 2016.

Cited by: 78|Bibtex|Views86|Links
EI
Keywords:
recurrent neural networkchinese poetry generationpoem qualitysong iambicsub topicMore(12+)
Weibo:
The poems generated by statistical machine translation are better in Poeticness than recurrent neural network language model, which demonstrates that the translation based method can better capture the mapping relation between two adjacent lines

Abstract:

Chinese poetry generation is a very challenging task in natural language processing. In this paper, we propose a novel two-stage poetry generating method which first plans the sub-topics of the poem according to the useru0027s writing intent, and then generates each line of the poem sequentially, using a modified recurrent neural network ...More

Code:

Data:

0
Introduction
  • The classical Chinese poetry is a great and important heritage of Chinese culture. During the history of more than two thousand years, millions of beautiful poems are written to praise heroic characters, beautiful scenery, love, friendship, etc.
  • The principles of a quatrain include: The poem consists of four lines and each line has five or seven characters; every character has a particular tone, Ping or Ze; the last character of the second and last line in a quatrain must belong to the same rhyme category (Wang, 2002)
  • With such strict restrictions, the well-written quatrain is full of rhythmic beauty
Highlights
  • The classical Chinese poetry is a great and important heritage of Chinese culture
  • Inspired by the observation that a human poet shall make an outline first before writing a poem, we propose a planning-based poetry generation approach (PPG) that first generates an outline according to the user’s writing intent and generates the poem
  • The poems generated by statistical machine translation (SMT) are better in Poeticness than recurrent neural network language model (RNNLM), which demonstrates that the translation based method can better capture the mapping relation between two adjacent lines
  • ANMT is a strong baseline which performs better than SMT, RNNLM and RNNPG, but lower than our approach
  • We proposed a novel two-stage poetry generation method which first explicitly decomposes the user’s writing intent into a series of sub-topics, and generates a poem iteratively using a modified attention based recurrent neural network (RNN) encoder-decoder framework
  • A comprehensive evaluation with human judgments demonstrates that our proposed approach outperforms the state-of-the-art poetry generating methods and the poem quality is somehow comparable to human poets
  • The evaluation by human experts shows that our approach outperforms all the baseline models and the poem quality is somehow comparable to human poets
Methods
  • 4.1 Dataset

    In this paper, the authors focus on the generation of Chinese quatrain which has 4 lines and each line has the same length of 5 or 7 characters.
  • All the poems in the training set are first segmented into words using a CRF based word segmentation system.
  • The word with the highest TextRank score is selected as the keyword for the line.
  • In this way, the authors can extract a sequence of 4 keywords for every quatrain.
  • From the training corpus of poems, the authors extracted 72,859 keyword sequences, which is used to train the RNN language model for keyword expansion.
  • For knowledge-based expansion, the authors use Baidu Baike1 and Wikipedia as the extra sources of knowledge
Results
  • The results of the human evaluation are shown in Table 4.
  • The poems generated by SMT are better in Poeticness than RNNLM, which demonstrates that the translation based method can better capture the mapping relation between two adjacent lines.
  • ANMT is a strong baseline which performs better than SMT, RNNLM and RNNPG, but lower than the approach.
  • Both ANMT and PPG use the attention based enc-dec framework.
Conclusion
  • The authors proposed a novel two-stage poetry generation method which first explicitly decomposes the user’s writing intent into a series of sub-topics, and generates a poem iteratively using a modified attention based RNN encoder-decoder framework.
  • The modified RNN enc-dec model has two encoders that can encode both the sub-topic and the preceding text.
  • The authors have demonstrated that using encyclopedias as an extra source of knowledge, the approach can expand users’ input into appropriate sub-topics for poem generation.
  • The authors will apply the approach to other forms of literary genres e.g. Song iambics, Yuan Qu etc., or poems in other languages
Summary
  • Introduction:

    The classical Chinese poetry is a great and important heritage of Chinese culture. During the history of more than two thousand years, millions of beautiful poems are written to praise heroic characters, beautiful scenery, love, friendship, etc.
  • The principles of a quatrain include: The poem consists of four lines and each line has five or seven characters; every character has a particular tone, Ping or Ze; the last character of the second and last line in a quatrain must belong to the same rhyme category (Wang, 2002)
  • With such strict restrictions, the well-written quatrain is full of rhythmic beauty
  • Methods:

    4.1 Dataset

    In this paper, the authors focus on the generation of Chinese quatrain which has 4 lines and each line has the same length of 5 or 7 characters.
  • All the poems in the training set are first segmented into words using a CRF based word segmentation system.
  • The word with the highest TextRank score is selected as the keyword for the line.
  • In this way, the authors can extract a sequence of 4 keywords for every quatrain.
  • From the training corpus of poems, the authors extracted 72,859 keyword sequences, which is used to train the RNN language model for keyword expansion.
  • For knowledge-based expansion, the authors use Baidu Baike1 and Wikipedia as the extra sources of knowledge
  • Results:

    The results of the human evaluation are shown in Table 4.
  • The poems generated by SMT are better in Poeticness than RNNLM, which demonstrates that the translation based method can better capture the mapping relation between two adjacent lines.
  • ANMT is a strong baseline which performs better than SMT, RNNLM and RNNPG, but lower than the approach.
  • Both ANMT and PPG use the attention based enc-dec framework.
  • Conclusion:

    The authors proposed a novel two-stage poetry generation method which first explicitly decomposes the user’s writing intent into a series of sub-topics, and generates a poem iteratively using a modified attention based RNN encoder-decoder framework.
  • The modified RNN enc-dec model has two encoders that can encode both the sub-topic and the preceding text.
  • The authors have demonstrated that using encyclopedias as an extra source of knowledge, the approach can expand users’ input into appropriate sub-topics for poem generation.
  • The authors will apply the approach to other forms of literary genres e.g. Song iambics, Yuan Qu etc., or poems in other languages
Tables
  • Table1: An example of Tang poetry. The tone is shown at the end of each line. P represents the leveltone, and Z represents the downward-tone; * indicates that the tone can be either. The rhyming characters are in boldface
  • Table2: Training triples extracted from the quatrain in Table 1
  • Table3: Evaluation standards in human judgement
  • Table4: Human evaluation results of all the systems. Diacritics ∗∗ (p <0.01) and ∗ (p <0.05) indicate that our model (PPG) is significantly better than all other systems
  • Table5: Blind test to distinguish Human-written Poems (HP) from Machine-generated Poems (MP)
  • Table6: A pair of poems selected from the blind test. The left one is a machine-generated poem, and the right one is written by Shaoti Ge, a poet lived in the Song Dynasty
  • Table7: Examples of poems generated from titles of modern concepts
Download tables as Excel
Related work
  • Poetry generation is a challenging task in NLP. Oliveira (2009; 2012) proposed a Spanish poem generation method based on semantic and grammar templates. Netzer et al (2009) employed a method based on word association measures. Tosa et al (2008) and Wu et al (2009) used a phrase search approach for Japanese poem generation. Greene et al (2010) applied statistical methods to analyze, generate and translate rhythmic poetry. Colton et al (2012) described a corpus-based poetry generation system that uses templates to construct poems according to the given constrains. Yan et al (2013) considered the poetry generation as an optimization problem based on a summarization framework with several constraints. Manurung (2004; 2012) and Zhou et al (2010) used genetic algorithms for generating poems. An important approach to poem generation is based on statistical machine translation (SMT). Jiang and Zhou (2008) used an SMT-based model in generating Chinese couplets which can be regarded as simplified regulated verses with only two lines. The first line is regarded as the source language and translated into the second line. He et al (2012) extended this method to generate quatrains by translating the previous line to the next line sequentially.
Funding
  • This research was supported by the National Basic Research Program of China (973 program No 2014CB340505), the National Key Research and Development Program of China (Grant No 2016YFB1000904), the National Science Foundation for Distinguished Young Scholars of China (Grant No 61325010) and the Fundamental Research Funds for the Central Universities of China (Grant No WK2350000001)
Reference
  • Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
    Findings
  • Sergey Brin and Lawrence Page. 1998. The anatomy of a large-scale hypertextual web search engine. Computer Networks, 30:107–117.
    Google ScholarLocate open access versionFindings
  • Kyunghyun Cho, Bart Van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
    Findings
  • Simon Colton, Jacob Goodwin, and Tony Veale. 2012. Full-face poetry generation. In ICCC.
    Google ScholarFindings
  • Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850.
    Findings
  • Erica Greene, Tugba Bodrumlu, and Kevin Knight. 2010. Automatic analysis of rhythmic poetry with applications to generation and translation. In EMNLP.
    Google ScholarFindings
  • Jing He, Ming Zhou, and Long Jiang. 2012. Generating chinese classical poems with statistical machine translation models. In Twenty-Sixth AAAI Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
  • Long Jiang and Ming Zhou. 200Generating chinese couplets using a statistical mt approach. In Proceedings of the 22nd International Conference on Computational Linguistics-Volume 1, pages 377–384. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023.
    Findings
  • Ruli Manurung, Graeme Ritchie, and Henry Thompson. 2012. Using genetic algorithms to create meaningful poetic text. Journal of Experimental & Theoretical Artificial Intelligence, 24(1):43–64.
    Google ScholarLocate open access versionFindings
  • Hisar Manurung. 2004. An evolutionary algorithm approach to poetry generation.
    Google ScholarFindings
  • Yejin Choi Marjan Ghazvininejad, Xing Shi and Kevin Knight. 2016. Generating topical poetry. In EMNLP.
    Google ScholarFindings
  • Rada Mihalcea and Paul Tarau. 2004. Textrank: Bringing order into text. In EMNLP.
    Google ScholarFindings
  • Tomas Mikolov, Martin Karafiat, Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In INTERSPEECH, volume 2, page 3.
    Google ScholarLocate open access versionFindings
  • Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
    Findings
  • Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, and Zhi Jin. 20Sequence to backward and forward sequences: A content-introducing approach to generative short-text conversation. In Proceedings the 26th International Conference on Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Yael Netzer, David Gabay, Yoav Goldberg, and Michael Elhadad. 2009. Gaiku: Generating haiku with word associations norms. In Proceedings of the Workshop on Computational Approaches to Linguistic Creativity, pages 32–39. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Hugo Goncalo Oliveira. 2009. Automatic generation of poetry: an overview. Universidade de Coimbra.
    Google ScholarFindings
  • Hugo Goncalo Oliveira. 2012. Poetryme: a versatile platform for poetry generation. Computational Creativity, Concept Invention, and General Intelligence, 1:21.
    Google ScholarLocate open access versionFindings
  • Jost Schatzmann, Kallirroi Georgila, and Steve Young. 2005. Quantitative evaluation of user simulation techniques for spoken dialogue systems. In 6th SIGdial Workshop on DISCOURSE and DIALOGUE.
    Google ScholarLocate open access versionFindings
  • Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112.
    Google ScholarLocate open access versionFindings
  • Naoko Tosa, Hideto Obara, and Michihiko Minoh. 2008. Hitch haiku: An interactive supporting system for composing haiku poem. In Entertainment Computing-ICEC 2008, pages 209–216. Springer.
    Google ScholarLocate open access versionFindings
  • Alan M Turing. 1950. Computing machinery and intelligence. Mind, 59(236):433–460.
    Google ScholarLocate open access versionFindings
  • Qixin Wang, Tianyi Luo, Dong Wang, and Chao Xing. 2016. Chinese song iambics generation with neural attention-based model. CoRR, abs/1604.06274.
    Findings
  • Li Wang. 2002. A summary of rhyming constraints of chinese poems.
    Google ScholarFindings
  • Xiaofeng Wu, Naoko Tosa, and Ryohei Nakatsu. 2009. New hitch haiku: An interactive renku poem composition supporting tool applied for sightseeing navigation system. In Entertainment Computing–ICEC 2009, pages 191–196. Springer.
    Google ScholarLocate open access versionFindings
  • Rui Yan, Han Jiang, Mirella Lapata, Shou-De Lin, Xueqiang Lv, and Xiaoming Li. 2013. i, poet: Automatic chinese poetry composition through a generative summarization framework under constrained optimization. In IJCAI.
    Google ScholarFindings
  • Xiaoyuan Yi, Ruoyu Li, and Maosong Sun. 2016. Generating chinese classical poems with rnn encoder-decoder. CoRR, abs/1604.01537.
    Findings
  • Matthew D. Zeiler. 2012. Adadelta: An adaptive learning rate method. CoRR, abs/1212.5701.
    Findings
  • Xingxing Zhang and Mirella Lapata. 2014. Chinese poetry generation with recurrent neural networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 670–680, Doha, Qatar, October. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Cheng-Le Zhou, Wei You, and Xiaojun Ding. 2010. Genetic algorithm and its implementation of automatic generation of chinese songci. Journal of Software, 21(3):427–437.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments