Automatic text summarization using deep reinforced model coupling contextualized word representation and attention mechanism

With the rapid and unprecedented growth of textual data in recent years, there is a remarkable need for automatic text summarization models to retrieve useful information from these large numbers of textual documents without human intervention within a reasonable time. Text summarization is commonly performed based on extractive and abstractive paradigms. Although different machine learning and deep learning based methods have been proposed for the task of text summarization during the last decades, they are still in their early steps of development and their potential has yet to be fully explored. Accordingly, a new summarization model is proposed in this paper which takes advantage of both extractive and abstractive text summarization models as a single unified model based on the strategy gradient of reinforcement learning. The proposed model also employs the combination of convolutional neural network and gated recurrent unit in both extraction and abstraction modules besides attention mechanism. Moreover, language models, namely Word2Vec and BERT, are used as the backbone of the proposed model to better express sentence semantics as a word vector. We conducted our experiments on widely-studied text summarization datasets (CNN\Daily Mail and DUC-2004) and according to the empirical results, not only the proposed model achieved higher accuracy compared to both extractive and abstractive summarization models in terms of ROUGE metric but also its generated summaries presented higher saliency and readability based on human evaluation.