We apologize for a recent technical issue with our email system, which temporarily affected account activations. Accounts have now been activated. Authors may proceed with paper submissions. PhDFocusTM
CFP last date
20 December 2024
Reseach Article

Keyphrase based Evaluation of Automatic Text Summarization

by Fatma Elghannam, Tarek El-shishtawy
International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 117 - Number 7
Year of Publication: 2015
Authors: Fatma Elghannam, Tarek El-shishtawy
10.5120/20564-2953

Fatma Elghannam, Tarek El-shishtawy . Keyphrase based Evaluation of Automatic Text Summarization. International Journal of Computer Applications. 117, 7 ( May 2015), 5-8. DOI=10.5120/20564-2953

@article{ 10.5120/20564-2953,
author = { Fatma Elghannam, Tarek El-shishtawy },
title = { Keyphrase based Evaluation of Automatic Text Summarization },
journal = { International Journal of Computer Applications },
issue_date = { May 2015 },
volume = { 117 },
number = { 7 },
month = { May },
year = { 2015 },
issn = { 0975-8887 },
pages = { 5-8 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume117/number7/20564-2953/ },
doi = { 10.5120/20564-2953 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2024-02-06T22:58:40.804751+05:30
%A Fatma Elghannam
%A Tarek El-shishtawy
%T Keyphrase based Evaluation of Automatic Text Summarization
%J International Journal of Computer Applications
%@ 0975-8887
%V 117
%N 7
%P 5-8
%D 2015
%I Foundation of Computer Science (FCS), NY, USA
Abstract

The development of methods to deal with the informative contents of the text units in the matching process is a major challenge in automatic summary evaluation systems that use fixed n-gram matching. The limitation causes inaccurate matching between units in a peer and reference summaries. The present study introduces a new Keyphrase based Summary Evaluator (KpEval) for evaluating automatic summaries. The KpEval relies on the keyphrases since they convey the most important concepts of a text. In the evaluation process, the keyphrases are used in their lemma form as the matching text unit. The system was applied to evaluate different summaries of Arabic multi-document data set presented at TAC2011. The results showed that the new evaluation technique correlates well with the known evaluation systems: Rouge-1, Rouge-2, Rouge-SU4, and AutoSummENG–MeMoG. KpEval has the strongest correlation with AutoSummENG–MeMoG, Pearson and spearman correlation coefficient measures are 0. 8840, 0. 9667 respectively.

References
  1. El-Shishtawy, T. , & El-Ghannam, F. 2012. An Accurate Arabic Root-Based Lemmatizer for Information Retrieval Purposes. International Journal of Computer Science Issues (IJCSI), 9(1). ?
  2. El-Shishtawy, T. , & El-Ghannam, F. 2012, May. Keyphrase based Arabic summarizer (KPAS). Informatics and Systems (INFOS), 2012 8th International Conference. (pp. NLP-7. IEEE). ?
  3. Giannakopoulos, G. , Karkaletsis, V. , Vouros, G. , & Stamatopoulos, P. 2008. Summarization system evaluation revisited: N-gram graphs. ACM Transactions on Speech and Language Processing (TSLP), 5(3), 5. ?
  4. Hovy, E. , Lin, C. Y. , & Zhou, L. 2005. Evaluating duc 2005 using basic elements. In Proceedings of DUC (Vol. 2005). ?
  5. Hovy, E. , Lin, C. Y. , Zhou, L. , & Fukumoto, J. 2006, May. Automated summarization evaluation with basic elements. In Proceedings of the Fifth Conference on Language Resources and Evaluation (LREC 2006) (pp. 604-611). ?
  6. Jones, K. S. , & Galliers, J. R. (Eds. ). 1996. Evaluating natural language processing systems: An analysis and review (Vol. 1083). Springer. ?
  7. Lin, C. Y. 2004, July. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop (pp. 74-81). ?
  8. Lin, C. Y. , & Hovy, E. 2003, May. Automatic evaluation of summaries using n-gram co-occurrence statistics. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1 (pp. 71-78). Association for Computational Linguistics. ?
  9. Papineni, K. , Roukos, S. , Ward, T. , & Zhu, W. J. 2002, July. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics (pp. 311-318). Association for Computational Linguistics. ?
  10. Tratz, S. , & Hovy, E. 2008. Summarization evaluation using transformed basic elements. In Proceedings of the 1st Text Analysis Conference. ?
Index Terms

Computer Science
Information Sciences

Keywords

Evaluating automatic text summarization keyphrase-based summary evaluation Summarization keyphrase extraction Arabic summary evaluation.