Deep Learning Architecture for Hybrid Multi-Document Abstractive Summarization using Sentence Embeddings
DOI:
https://doi.org/10.26438/ijcse/v8i4.59Keywords:
Multi-Document Summarization, Abstractive, Skip-thought embedding, ROUGEAbstract
Multi-document summarization aims at generating a comprehensive summary of multiple documents related to a common topic without repeatedly conveying the same piece of information while covering the essential information from all the documents. Extractive summarization methods exist to handle Multi-document summarization, while the Abstractive summarization methods are limited to handling single-document summaries. This paper proposes abstractive summarization of multiple documents by extending the state-of-the-art single-document abstractive summarization model Pointer-Generator to generate a multi-document summary. The short abstract summaries generated upon multiple applications of the Pointer-Generator model on individual documents are clustered at the sentence level using Skip-thought embeddings. The representative sentences from each of the clusters constitute the final summary in order to avoid similar sentences while generating the multi-document abstractive summary without loss of information. The proposed methodology is evaluated using the DUC2004 benchmark dataset and observed a gain of 2 to 7 points of ROUGE scores compared to existing state of the art methods.
References
[1] Chen, CL Philip, and Chun-Yang Zhang. "Data-intensive applications, challenges, techniques and technologies: A survey on Big Data." Information sciences 275 (2014): 314-347.
[2] Raghupathi, Wullianallur, and Viju Raghupathi. "Big data analytics in healthcare: promise and potential." Health information science and systems 2.1 (2014): 3.
[3] Andreu-Perez, Javier, et al. "Big data for health." IEEE journal of biomedical and health informatics 19.4 (2015): 1193-1208.
[4] Singh, Anita Kumari, and Mogalla Shashi. "Deep Learning Architecture for Multi-Document Summarization as a cascade of Abstractive and Extractive Summarization approaches." International Journal of Computer Sciences and Engineering 7.3 (2019): 950-954.
[5] Kiros, Ryan, et al. "Skip-thought vectors." Advances in neural information processing systems. 2015.
[6] Saggion, Horacio, and Thierry Poibeau. "Automatic text summarization: Past, present, and future." Multi-source, multilingual information extraction, and summarization. Springer, Berlin, Heidelberg, 2013. 3-21.
[7] Chen, Kuan-Yu, et al. "Extractive broadcast News summarization leveraging recurrent neural network language modeling techniques." IEEE Transactions on Audio, Speech, and Language Processing 23.8 (2015): 1322-1334.
[8] Cheng, Jianpeng, and Mirella Lapata. "Neural summarization by extracting sentences and words." arXiv preprint arXiv:1603.07252 (2016).
[9] Verma, Sukriti, and Vagisha Nidhi. "Extractive summarization using deep learning." arXiv preprint arXiv:1708.04439 (2017).
[10] Khan, Atif, and Naomie Salim. "A review of abstractive summarization methods." Journal of Theoretical and Applied Information Technology 59.1 (2014): 64-72.
[11] Nallapati, Ramesh, et al. "Abstractive text summarization using sequence-to-sequence rnns and beyond." arXiv preprint arXiv:1602.06023 (2016).
[12] Pasunuru, Ramakanth, Han Guo, and Mohit Bansal. "Towards improving abstractive summarization via entailment generation." Proceedings of the Workshop on New Frontiers in Summarization. 2017.
[13] Luong, Minh-Thang, et al. "Multi-task sequence to sequence learning." arXiv preprint arXiv:1511.06114 (2015).
[14] Ling, Jeffrey. Coarse-to-fine attention models for document summarization. Diss. 2017.
[15] Rush, Alexander M., Sumit Chopra, and Jason Weston. "A neural attention model for abstractive sentence summarization." arXiv preprint arXiv:1509.00685 (2015).
[16] Shen, Shi-Qi, et al. "Recent advances on neural headline generation." Journal of computer science and technology 32.4 (2017): 768-784.
[17] See, Abigail, Peter J. Liu, and Christopher D. Manning. "Get to the point: Summarization with Pointer-Generator networks." arXiv preprint arXiv:1704.04368 (2017).
[18] Singh, Anita Kumari, and Mogalla Shashi. "Vectorization of Text Documents for Identifying Unifiable News Articles." corpora 10.7 (2019).
[19] Hermann, Karl Moritz, et al. "Teaching machines to read and comprehend." Advances in neural information processing systems. 2015.
[20] Lin, Chin-Yew. "Rouge: A package for automatic evaluation of summaries." Text summarization branches out. 2004.
[21] Erkan, Günes, and Dragomir R. Radev. "Lexrank: Graphbased lexical centrality as salience in text summarization." Journal of artificial intelligence research 22 (2004): 457-479. [7]
[22] Carbonell, Jaime G., and Jade Goldstein. "The use of MMR, diversity-based reranking for reordering documents and producing summaries." SIGIR. Vol. 98. 1998.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors contributing to this journal agree to publish their articles under the Creative Commons Attribution 4.0 International License, allowing third parties to share their work (copy, distribute, transmit) and to adapt it, under the condition that the authors are given credit and that in the event of reuse or distribution, the terms of this license are made clear.
