Automatic essay grading with probabilistic latent semantic analysis
Published 1 January 2005Open access
Tuomo Kakkonen, Niko Myller, Jari Timonen, Erkki Sutinen
Citations57
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The results comparing PLSA and LSA with three essay sets from various subjects found the methods were found to be almost equal in the accuracy measured by Spearman correlation between the grades given by the system and a human.
Abstract
Probabilistic Latent Semantic Analysis (PLSA) is an information retrieval technique proposed to improve the problems found in Latent Semantic Analysis (LSA). We have applied both LSA and PLSA in our system for grading essays written in Finnish, called Automatic Essay Assessor (AEA). We report the results comparing PLSA and LSA with three essay sets from various subjects. The methods were found to be almost equal in the accuracy measured by Spearman correlation between the grades given by the system and a human. Furthermore, we propose methods for improving the usage of PLSA in essay grading.
Keywords
Computer Science
Journal of the Royal Statistical Society Series B (Statistical Methodology)Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
49,657 Citations1977A. P. Dempster, N. M. Laird +1 more
Journal of Machine Learning ResearchLatent dirichlet allocation
27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more
Journal of the American Society for Information ScienceIndexing by latent semantic analysis
12,677 Citations1990Scott Deerwester, Susan Dumais +3 more
Discourse ProcessesAn introduction to latent semantic analysis
4,817 Citations1998Thomas K. Landauer, Peter W. Foltz +1 more
The adequacy of LSA's reflection of human knowledge has been established in a variety of ways, for example, its scores overlap those of humans on standard vocabulary and subject matter tests; it mimics human word sorting and category judgments; it simulates word‐word and passage‐word lexical priming data.
Machine LearningUnsupervised Learning by Probabilistic Latent Semantic Analysis
2,443 Citations2001Thomas Hofmann
This paper proposes to make use of a temperature controlled version of the Expectation Maximization algorithm for model fitting, which has shown excellent performance in practice, and results in a more principled approach with a solid foundation in statistical inference.
How Well Can Passage Meaning be Derived without Using Word Order? A Comparison of Latent Semantic Analysis and Humans
274 Citations1997Thomas K. Landauer, Darrell Laham +2 more
An exploratory approach was provided by asking humans to judge the quality and quantity of knowledge conveyed by short student essays on scientific topics and comparing the interrater reliability and predictive accuracy of their estimates with the performance of a corpus-based statistical model that takes no account of word order within an essay.
On an equivalence between PLSI and LDA
200 Citations2003Mark Girolami, Ata Kabán
PLSI is a maximum a posteriori estimated LDA model under a uniform Dirichlet prior, therefore the perceived shortcomings of PLSI can be resolved and elucidated within the LDA framework.
IEEE Intelligent Systems and their ApplicationsThe debate on automated essay grading
172 Citations2000Marti A. Hearst
The director of the Natural Language Processing group at Educational Testing Service, Kukich, provides an insider's view of the history of the field of automated essay grading and describes how ETS is currently using computer programs to supplement human judges in the grading process.
Phi Delta KappanThe Computer Moves into Essay Grading: Updating the Ancient Test.
150 Citations1995Ellis B. Page, Nancy S. Petersen
A blind test has demonstrated that a computer can simulate the judgment of a group of human judges on a brand-new set of essays, and the bottom line is that the computer program did predict human judgments well - perhaps even better than three human judges.
Journal of Educational Computing ResearchA System to Assess the Semantic Content of Student Essays
98 Citations2001Benoı̂t Lemaire, Philippe Dessus
A system that can automatically assess a student essay based on its content using Latent Semantic Analysis, a tool which is used to represent the meaning of words as vectors in a high-dimensional space.
Interactive Learning EnvironmentsSelect-a-Kibitzer: A Computer Tool that Gives Meaningful Feedback on Student Compositions
86 Citations2000Peter Hastings, Arthur C. Graesser
The composition research that forms the basis of the projec t, and the interaction and implementation of Select-a-Kibitzer are described, which focuses on techniques for using LSA to provide feedback about the meaning of the composition.
Comparison of Dimension Reduction Methods for Automated Essay Grading
84 Citations2008Tuomo Kakkonen, Niko Myller +2 more
The results show that the use of learning materials as training data for the grading model outperforms the k-NN-based grading methods and the division of the learning materials in the training data is crucial.
Assessment & Evaluation in Higher EducationOn-line Grading of Student Essays: PEG goes on the World Wide Web
52 Citations2001Mark D. Shermis, Howard R. Mzumara +2 more
The feasibility of employing Project Essay Grade software to evaluate web-based student essays that serve as placement tests at a large Mid-Western university is examined and PEG would appear to be a cost-effective means of grading written work of this type.
Automatic assessment of the content of essays based on course materials
38 Citations2005Tuomo Kakkonen, Erkki Sutinen
A method for evaluating the content of essays based on latent semantic analysis (LSA), which was originally developed for information retrieval and has successfully been applied for assessment of essays in several applications is presented.
Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval - SIGIR '03On an equivalence between PLSI and LDA
29 Citations2003Mark Girolami, Ata Kabán
Approximate Natural Language Understanding for an Intelligent Tutor
14 Citations1999Peter Hastings, Katja Wiemer-Hastings +1 more
This paper describes the tutoring system and Latent Semantic Analysis, and how they operate together, and describes the evaluation of LSA’s performance by comparing its judgments with those of human raters.
Benefits of Modularity in an Automated Essay Scoring System
13 Citations2000Jill Burstein, Daniel Marcu
How additional features from rhetorical parse trees were integrated into e-rater and how the salience of automatically generated discourse-based essay summaries was evaluated for use as instructional feedback through the re-use of e- rater's topical analysis module are discussed.
Semi-Automatic Evaluation Features in Computer-assisted Essay Assessment.
9 Citations2004Tuomo Kakkonen, Niko Myller +1 more
This work presents a system that evaluates the content of essays based on Latent Semantic Analysis (LSA), which applies LSA to compare the conceptual similarity between the essays and selected text passages from the course material covering the essay assignment-specific subject matter.