An Automated Summarization Assessment Algorithm for Identifying Summarizing Strategies
PLoS ONEPublished 6 January 2016Open access
Asad Abdi, Norisma Idris, Rasim Alguliyev, Ramiz M. Aliguliyev
Citations13
SJR quartileQ1
SJR score0.80
SNIP1.07
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper proposes an algorithm based on the combination of semantic relations between words and their syntactic composition to identify summarizing strategies employed by students in summary writing and implements the algorithm for the automated summarization assessment system.
Abstract
This paper aims to propose an algorithm based on the combination of semantic relations between words and their syntactic composition to identify summarizing strategies employed by students in summary writing. An innovative aspect of our algorithm lies in its ability to identify summarizing strategies at the syntactic and semantic levels. The efficiency of the algorithm is measured in terms of Precision, Recall and F-measure. We then implemented the algorithm for the automated summarization assessment system that can be used to identify the summarizing strategies used by students in summary writing.
Keywords
Computer Science
BiometricsThe Measurement of Observer Agreement for Categorical Data
78,934 Citations1977J. Richard Landis, Gary G. Koch
A general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies is presented and tests for interobserver bias are presented in terms of first-order marginal homogeneity and measures of interob server agreement are developed as generalized kappa-type statistics.
Journal of the American Society for Information ScienceIndexing by latent semantic analysis
12,677 Citations1990Scott Deerwester, Susan Dumais +3 more
Psychological BulletinWeighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit.
8,537 Citations1968Jacob Cohen
The Kw provides for the incorpation of ratio-scaled degrees of disagreement (or agreement) to each of the cells of the k * k table of joi.
ROUGE: A Package for Automatic Evaluation of Summaries
8,286 Citations2004Chin-Yew Lin
Four different RouGE measures are introduced: ROUGE-N, ROUge-L, R OUGE-W, and ROUAGE-S included in the Rouge summarization evaluation package and their evaluations.
Psychological ReviewToward a model of text comprehension and production.
5,035 Citations1978Walter Kintsch, Teun A. van Dijk
The semantic structure of texts can be described both at the local microlevel and at a more global macrolevel, and a model for text comprehension based on this notion accounts for the formation of a coherent semantic text base in terms of a cyclical process constrained by limitations of working memory.
An Information-Theoretic Definition of Similarity
3,662 Citations1998Dekang Lin
This work presents an informationtheoretic definition of similarity that is applicable as long as there is a probabilistic model and demonstrates how this definition can be used to measure the similarity in a number of different domains.
Generating Typed Dependency Parses from Phrase Structure Parses
2,304 Citations2006Marie-Catherine de Marneffe, Bill MacCartney +1 more
A system for extracting typed dependency parses of English sentences from phrase structure parses that captures inherent relations occurring in corpus texts that can be critical in real-world applications is described.
A trainable document summarizer
1,349 Citations1995Julian Kupiec, Jan Pedersen +1 more
The trends in the results are in agreement with those of Edmundson who used a subjectively weighted combination of features as opposed to training the feature weights using a corpus, which suggests that even shorter extracts may be useful indicative summmies.
Corpus-based and knowledge-based measures of text semantic similarity
1,189 Citations2006Rada Mihalcea, Courtney D. Corley +1 more
This paper shows that the semantic similarity method out-performs methods based on simple lexical matching, resulting in up to 13% error rate reduction with respect to the traditional vector-based similarity metric.
IEEE Transactions on Knowledge and Data EngineeringSentence similarity based on semantic nets and corpus statistics
821 Citations2006Yuhua Li, David McLean +3 more
Experiments demonstrate that the proposed method provides a similarity measure that shows a significant correlation to human intuition and can be used in a variety of applications that involve text knowledge representation and discovery.
Journal of Verbal Learning and Verbal BehaviorMacrorules for summarizing texts: the development of expertise
697 Citations1983Ann L. Brown, Jeanne D. Day
Journal of Emerging Technologies in Web IntelligenceA Survey of Text Summarization Extractive Techniques
606 Citations2010Vishal Gupta, Gurpreet Singh Lehal
A Survey of Text Summarization Extractive techniques has been presented and it is shown that extracting important sentences, paragraphs etc. from the source text and concatenating them into shorter form conveys the most important information from the original text document.
Empirical Studies on the Disambiguation of Cue Phrases
372 Citations1991Julia Hirschberg, Diane Litman
This paper reports results of empirical studies on discourse and sentential uses of cue phrases, in which both text-based and prosodic features were examined for disambiguating power.
Computers & EducationDevelopment of an adaptive learning system with two sources of personalization information
328 Citations2007Judy C. R. Tseng, Hui‐Chun Chu +2 more
By analyzing the results from three groups of students using different adaptive learning approaches, it can be found that the innovative approach is helpful in improving both the learning achievement and learning efficiency of individual students.
Expert Systems with ApplicationsA new sentence similarity measure and sentence based extractive technique for automatic text summarization
266 Citations2008Ramiz M. Aliguliyev
The purpose of present paper is to show, that summarization result not only depends on optimized function, and also depends on a similarity measure.
Computer Speech & LanguageGA, MR, FFNN, PNN and GMM based models for automatic text summarization
249 Citations2008Mohamed I. Abdel‐Fattah, Fuji Ren
This work proposes an approach to address the problem of improving content selection in automatic text summarization by using some statistical tools, which takes into account several features, including sentence position, positive keyword, negative keyword, sentence centrality, sentence resemblance to the title, sentenceclusion of name entity, sentence inclusion of numerical data, sentence relative length and aggregated similarity.
Learning to Grade Short Answer Questions using Semantic Similarity Measures and Dependency Graph Alignments
248 Citations2011Michael Mohler, Răzvan Bunescu +1 more
This work combines several graph alignment features with lexical semantic similarity measures using machine learning techniques and shows that the student answers can be more accurately graded than if the semantic measures were used in isolation.
Lecture notes in computer scienceThe Evaluation of Sentence Similarity Measures
218 Citations2008Palakorn Achananuparp, Xiaohua Hu +1 more
This work evaluated fourteen existing text similarity measures which have been used to calculate similarity score between sentences in many text applications, and found three of them to be inadequate.
Sentence Extraction as a Classification Task
217 Citations1997Simone Teufel
Swarm and Evolutionary ComputationSentence selection for generic document summarization using an adaptive differential evolution algorithm
84 Citations2011Rasim M. Аlguliev, Ramiz M. Aliguliyev +1 more
A document summarization model which extracts key sentences from given documents while reducing redundant information in the summaries is presented, and an innovative aspect of the model lies in its ability to remove redundancy while selecting representative sentences.
Automatic evaluation of students' answers using syntactically enhanced LSA
84 Citations2003Dharmendra Kanejiya, Arun Kumar +1 more
Syntactically Enhanced LSA is presented here an approach which generalizes LSA by considering a word along with its syntactic neighborhood given by the part-of-speech tag of its preceding word, as a unit of knowledge representation, which provides better discrimination of syntactic-semantic knowledge representation than LSA.
Topics in Language DisordersSummarizing Expository Texts
84 Citations2010Carol Westby, Barbara Culatta +2 more
eScholarship (California Digital Library)Rules for Syntax, Vectors for Semantics
77 Citations2001Peter Hastings, Iraide Zipitria
A model of language understanding that combines information from rule-based syntactic processing with a vector-based se- mantic representation which is learned from a corpus is evaluated as a cognitive model, and as a potential technique for natural language understanding.
Utilizing semantic word similarity measures for video retrieval
72 Citations2008Yusuf Aytar, Mubarak Shah +1 more
The method for retrieving a new concept is an unsupervised method, it outperforms the trained concept detectors on 7 out of 20 test concepts, and overall it performs very close to the trained detectors.
Computers & EducationInstructional effectiveness of a computer-supported program for teaching reading comprehension strategies
71 Citations2012Héctor R. Ponce, Mario López +1 more
The study involved 1041 fourth-grade elementary students from 21 schools distributed in three regions in central Chile, and participant teachers integrated this program into the Spanish language curriculum, instructing their students during thirty sessions of 90 min each during one school semester.
Proceedings of the ... IEEE International Conference on Acoustics, Speech, and Signal ProcessingUsing corpus and knowledge-based similarity measure in Maximum Marginal Relevance for meeting summarization
66 Citations2008Shasha Xie, Yang Liu
This paper introduces a corpus- based measure to capture the similarity at the semantic level, and compares this method with cosine similarity and centroid score that only considers the salient words in the segments.
Expert Systems with ApplicationsPDLK: Plagiarism detection using linguistic knowledge
64 Citations2015Asad Abdi, Norisma Idris +2 more
This paper presents a method to detect external plagiarism using the integration of semantic relations between words and their syntactic composition and shows that the proposed method is able to improve the performance compared with the participating systems in PAN-PC-11.
Automatic assessment of students’ free-text answers underpinned by the combination of a BLEU-inspired algorithm and latent semantic analysis
60 Citations2005Diana Pérez-Marín, Alfio Gliozzo +4 more
This is an electronic version of the paper presented at the International Florida Artificial Intelligence Research Society Conference, FLAIRS 2005 about the design and implementation of convolutional neural networks for knowledge representation.
Computers & EducationThe effectiveness of automatic text summarization in mobile learning contexts
58 Citations2013Guangbing Yang, Nian‐Shing Chen +4 more
The findings of this work indicate that properly summarized learning content is not only able to satisfy learning achievements, but also able to align content size with the unique characteristics and affordances of mobile devices.
Adding syntactic information to LSA
53 Citations2000Peter Hastings
An attempt to evaluate the benefits of providing LSA with thematic role information which comes from syntactic knowledge.
Computer Speech & LanguageImproving supervised learning for meeting summarization using sampling and regression
49 Citations2009Shasha Xie, Yang Liu
This paper reframe the extractive summarization task using a regression scheme instead of binary classification, and evaluates the approaches using the ICSI meeting corpus on both the human transcripts and speech recognition output, and shows performance improvement using different sampling methods and regression model.
Evaluating the effectiveness of features and sampling in extractive meeting summarization
47 Citations2008Shasha Xie, Yang Liu +1 more
The experimental results on both the human transcripts and recognition output, evaluated by the ROUGE summarization metrics, show that feature selection and data resampling help improve the system performance.
Asian Social ScienceThe Effect of Comprehension Strategy Instruction on EFL Learners’ Reading Comprehension
43 Citations2010Yen-Chi Fan
The findings of the study suggest that implementing comprehension strategy instruction for one semester may help learners adopt some degree of strategic reading behaviours, but it takes long-term efforts and practices for EFL learners to fully develop their strategic reading abilities.
Soft ComputingQuery-based multi-documents summarization using linguistic knowledge and content word expansion
36 Citations2015Asad Abdi, Norisma Idris +2 more
A query-based summarization method, which uses a combination of semantic relations between words and their syntactic composition, to extract meaningful sentences from document sets is introduced and demonstrates better performance as compared to other existing techniques on DUC 2005 and DUC 2006 datasets.
Information Processing & ManagementAutomatic summarization assessment through a combination of semantic and syntactic information for intelligent educational systems
35 Citations2015Asad Abdi, Norisma Idris +2 more
A specialized method that works well in assessing short summaries that integrates the semantic relations between words, and their syntactic composition and is able to obtain high accuracy and improve the performance compared with the current techniques.
Lecture notes in computer scienceData Warehousing and Knowledge Discovery
30 Citations2010Torben Bach Pedersen, Mukesh Mohania +2 more
A Cue-based Hub-Authority Approach for Multi-Document Text Summarization
29 Citations2006J. Zhanq, Le Sun +1 more
A new approach under the hub-authority framework is proposed that combines the text content with some cues and explores the sub-topics in the multi-documents by bringing the features of these sub- topics into graph-based sentence ranking algorithms.
CogPrints (University of Southampton)Computational Cognitive Models of Summarization Assessment Skills
29 Citations2005Guy Denhière, Philippe Dessus +2 more
A general computational cognitive model of the way a summary is assessed by teachers is presented, based on models of two subprocesses: determining the importance of sentences and guessing the cognitive rules that the student may have used.
A Semantic Summarization System: University of Birmingham at TAC 2008
21 Citations2008Abdullah Bawakid, Mourad Oussalah
A new query-based extractive summary methodology is put forward that makes use of phrasal decomposition of the text where each sentence is ascribed a scoring function, which will then be used to identify the most relevant sentences in the sequel.
Computers & EducationIntegrating reviewing strategies into shared electronic note-taking: Questioning, summarizing and note reading
19 Citations2013Chiung‐Hui Chiu, Chiu-Yi Wu +1 more
The results revealed that the SN-SS students' notes contained more facts and concepts than theSN-SR students', and the SN/SQ students achieved better grades on the unit quizzes and overall assessment two weeks later than those in the Sn-SR condition.
International Journal of Human-Computer StudiesHAUSS: Incrementally building a summarizer combining multiple techniques
18 Citations2014Filippo Galgani, Paul Compton +1 more
This paper presents HAUSS: a framework to quickly build specialized summarizers, integrating several base techniques into a single approach, which outperforms existing state-of-the-art general-purpose summarizers and machine learning approaches.
Lecture notes in computer scienceFrom Human to Automatic Summary Evaluation
14 Citations2004Iraide Zipitria, Jon A. Elorriaga +2 more
The aim of this study is to produce the design for a fully automatic summary evaluation system that could stand for human-like summarisation assessment, and an empirical study has been carried out to identify underlying cognitive processes.
Lecture notes in computer scienceData Warehousing and Knowledge Discovery
12 Citations2009Torben Bach Pedersen, Mukesh Mohania +2 more
Measuring the similarity of short texts by word similarity and tree kernels
11 Citations2010Yun Tian, Haisheng Li +2 more
A novel modeling method is presented to measure the similarity between short texts that takes word similarity feature and syntactic feature into account and can be used in a variety of applications involving automatic document summarization, text knowledge representation and discovery.
Computer and Information ScienceA Summary Sentence Decomposition Algorithm for Summarizing Strategies Identification
7 Citations2009Norisma Idris, Sapiyan Baba +1 more
An algorithm to identify the summarizing strategies employed by students using summary sentence decomposition is developed based on a set of heuristic rules.