Chinese Journal of Computers   Full Text
Title	Evaluation Model of Summary Coherence Based on Basic Element
Authors	LIU De-Xi1),2) JI Dong-Hong3)
Address	1)(School of Information Management, Jiangxi University of Finance & Economics, Nanchang 330013) 2)(Jiangxi Key Laboratory of Data and Knowledge Engineering, Jiangxi University of Finance and Economics, Nanchang 330013) 3)(School of Computer, Wuhan University, Wuhan 430079)
Year	2008
Issue	No.4(628—635)
Abstract &   Background	Abstract One of the key problems in automatic summarization is the absence of effective auto-evaluation method. A BE-Relation-Grid based evaluation model for "summary coherence" is proposed. BE(Basic Element) is viewed as the content unit and the "relation" part in BE as grammar role of the content unit. Then, the content coherence is scaled by BE relation transition probability in the BE-Relation-Grid. Experiment results on DUC2005 dataset show that the Pearson correlation coefficient of the evaluation results between this model and manual ones is 0.408, which increased by about 66% comparing with the result of entity grid model presented by Lapata in 2005. keywords automatic summarization; content coherence; basic element; relation grid background This work is supported by National Nature Science Foundation of China named by "A Study on Document Re-Ranking Techniques in Information Retrieval" (60703008) and "Sentence Ordering for Chinese Text Information Fusion" (60773011). To measure the performance of document re-ranking and sentence ordering techniques, a proper evaluation method should be selected. However, most of the existing automatic evaluation methods for information retrieval and automatic summarization are based on recall ratio of content. So, the work in this paper aims at exploring an auto-evaluation method of content’s readability.