International Journal of Computer Applications |
Foundation of Computer Science (FCS), NY, USA |
Volume 117 - Number 7 |
Year of Publication: 2015 |
Authors: Fatma Elghannam, Tarek El-shishtawy |
10.5120/20564-2953 |
Fatma Elghannam, Tarek El-shishtawy . Keyphrase based Evaluation of Automatic Text Summarization. International Journal of Computer Applications. 117, 7 ( May 2015), 5-8. DOI=10.5120/20564-2953
The development of methods to deal with the informative contents of the text units in the matching process is a major challenge in automatic summary evaluation systems that use fixed n-gram matching. The limitation causes inaccurate matching between units in a peer and reference summaries. The present study introduces a new Keyphrase based Summary Evaluator (KpEval) for evaluating automatic summaries. The KpEval relies on the keyphrases since they convey the most important concepts of a text. In the evaluation process, the keyphrases are used in their lemma form as the matching text unit. The system was applied to evaluate different summaries of Arabic multi-document data set presented at TAC2011. The results showed that the new evaluation technique correlates well with the known evaluation systems: Rouge-1, Rouge-2, Rouge-SU4, and AutoSummENG–MeMoG. KpEval has the strongest correlation with AutoSummENG–MeMoG, Pearson and spearman correlation coefficient measures are 0. 8840, 0. 9667 respectively.