An experimental evaluation of OCR text representations for learning document classifiers
International Journal on Document Analysis and Recognition (IJDAR)Published 1 July 1998
Markus Junker, Rainer Hoch
Citations24
SJR quartileQ1
SJR score0.83
SNIP1.96
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The results indicate that the use of n-grams is an attractive technique which can even compare to techniques relying on a morphological analysis, which holds for OCR texts as well as for correct ASCII texts.
Abstract
In the literature, many feature types are proposed for document classification. However, an extensive and systematic evaluation of the various approaches has not yet been done. In particular, evaluations on OCR documents are very rare. In this paper we investigate seven text representations based on n-grams and single words. We compare their effectiveness in classifying OCR texts and the corresponding correct ASCII texts in two domains: business letters and abstracts of technical reports. Our results indicate that the use of n-grams is an attractive technique which can even compare to techniques relying on a morphological analysis. This holds for OCR texts as well as for correct ASCII texts.
Keywords
Computer Science
N-gram-based text categorization
1,503 Citations1994William B. Cavnar, John M. Trenkle
An N-gram-based approach to text categorization that is tolerant of textual errors is described, which worked very well for language classification and worked reasonably well for classifying articles from a number of different computer-oriented newsgroups according to subject.
ACM Transactions on Information SystemsAutomated learning of decision rules for text categorization
867 Citations1994Chidanand Apté, Fred J. Damerau +1 more
It is shown that machine-generated decision rules appear comparable to human performance, while using the identical rule-based representation, and compared with other machine-learning techniques.
Journal of the ACMAutomatic Indexing: An Experimental Inquiry
552 Citations1961M. E. Maron
The design, execution and evaluation of a modest experimental study aimed at testing empirically one statistical technique for automatic indexed documents according to their subject content are described.
ScienceGauging Similarity with <i>n</i> -Grams: Language-Independent Categorization of Text
535 Citations1995Marc Damashek
A language-independent means of gauging topical similarity in unrestricted text by combining information derived from n-grams with a simple vector-space technique that makes sorting, categorization, and retrieval feasible in a large multilingual collection of documents.
Journal of DocumentationA THEORETICAL BASIS FOR THE USE OF CO‐OCCURRENCE DATA IN INFORMATION RETRIEVAL
465 Citations1977C. J. van Rijsbergen
This paper provides a foundation for a practical way of improving the effectiveness of an automatic retrieval system by measuring the extent of the dependence between index terms and using it to construct a non‐linear weighting function.
A comparison of classifiers and document representations for the routing problem
451 Citations1995Hinrich Schütze, David A. Hull +1 more
This paper considers three classification techniques which have decision rules that are derived via explicit error minimization linear discriminant analysis, logistic regression, and neuraf networks, and finds that features based on latent semantic indexing are more effective for techniques such aslinear discriminant anaf-ysis and logistic regressors, which have no way to protect against overfitting.
Scholarworks (University of Massachusetts Amherst)Representation and Learning in Information Retrieval
361 Citations1991David Lewis
A new theoretical model for text classification systems, including systems for document retrieval, automated indexing, electronic mail filtering, and similar tasks, is introduced, suggesting that the poor statistical characteristics of a syntactic indexing phrase representation negate its desirable semantic characteristics.
Classifying news stories using memory based reasoning
261 Citations1992Brij Masand, Gordon S. Linoff +1 more
A method for classifying news stories using Memory Based Reasoning (MBR) a k-nearest neighbor method, that does not require manual topic definitions, that is effective in reducing the development time to implement classification systems involving large number of topics for the purpose of classification, message routing etc.
Context-sensitive learning methods for text categorization
253 Citations1996William W. Cohen, Yoram Singer
Cluster-based text categorization
121 Citations1995Makoto Iwayama, Takenobu Tokunaga
The “efficiency, effectiveness, and noise tolerance of this search strategy were confirmed to be better than those of a full search, a category-based search, and a cluster- based search with nonprobabilistic clustering.
Lecture notes in computer scienceApplying an existing machine learning algorithm to text categorization
39 Citations1996Isabelle Moulinier, Jean‐Gabriel Ganascia
This paper describes how an existing similarity-based learning algorithm, Charade, is applied to the text categorization problem and compares the results with those obtained using decision tree construction algorithms.
Munich Personal RePEc Archive (Ludwig Maximilian University of Munich)Acquaintance: Language-Independent Document Categorization by N-Grams
28 Citations1995Stephen Phillip Huffman
Aquaintance is the name of a novel vector-space n-gram technique for categorizing documents that is completely language-independent, highly garble-resistant, and computationally simple.