Evaluating the utility of statistical phrases and latent semantic indexing for text classification

Published 26 June 2003

Huiwen Wu, Dimitrios Gunopulos

DOI10.1109/icdm.2002.1184036

Citations21

TL;DR

This paper focuses on statistical techniques to extract both adjacent and window phrases from documents, discovering that the positive effect of adding phrase terms is very limited, if the authors have already achieved good performance using single-word terms, even when SVD/LSI is used as the dimensionality reduction method.

Abstract

The term-based vector space model is a prominent technique for retrieving textual information. In this paper we examine the usefulness of phrases as terms in vector-based document classification. We focus on statistical techniques to extract both adjacent and window phrases from documents. We discover that the positive effect of adding phrase terms is very limited, if we have already achieved good performance using single-word terms, even when SVD/LSI is used as the dimensionality reduction method.

Keywords

Computer Science

Lecture notes in computer scienceText categorization with Support Vector Machines: Learning with many relevant features

7,925 Citations1998Thorsten Joachims

SVMs achieve substantial improvements over the currently best performing methods and behave robustly over a variety of di-erent learning tasks, eliminating the need for manual parameter tuning.

IEEE Intelligent Systems and their ApplicationsSupport vector machines

6,931 Citations1998Marti A. Hearst, Susan Dumais +3 more

This issue's collection of essays should help familiarize readers with this interesting new racehorse in the Machine Learning stable, and give a practical guide and a new technique for implementing the algorithm efficiently.

A Comparative Study on Feature Selection in Text Categorization

4,766 Citations1997Yiming Yang, Jan Pedersen

DF thresholding, the simplest method with the lowest cost in computation, can be reliably used instead of IG or CHI when the computation of these measures are too expensive, and strong correlations between the DF, IG and CHI values of a term are found.

A re-examination of text categorization methods

2,660 Citations1999Yiming Yang, Xin Liu

The results show that SVM, kNN and LLSF signi cantly outperform NNet and NB when the number of positive training instances per category are small, and that all the methods perform comparably when the categories are over 300 instances.

View PDF

IEEE Transactions on Neural NetworksSupport vector machines for spam categorization

1,457 Citations1999Harris Drucker, Donghui Wu +1 more

The use of support vector machines in classifying e-mail as spam or nonspam is studied by comparing it to three other classification algorithms: Ripper, Rocchio, and boosting decision trees, which found SVM's performed best when using binary features.

Feature Selection for Unbalanced Class Distribution and Naive Bayes

432 Citations1999Dunja Mladenić, Marko Grobelnik

This paper describes an approach to feature subset selection that takes into account problem speciics and learning algorithm characteristics, and shows that considering domain and algorithm characteristics signiicantly improves the results of classiication.

Journal of the American Society for Information ScienceA theory of term importance in automatic text analysis

388 Citations1975Gerard Salton, Chul‐Su Yang +1 more

Most existing automatic content analysis and indexing techniques are based on word frequency characteristics applied largely in an ad hoc manner, but terms exhibiting high occurence frequencies in individual documents are often useful for high recall performance, whereas terms with low frequency in the whole collection are useful forhigh precision.

An analysis of statistical and syntactic phrases

210 Citations1997Mandar Mitra, Chris Buckley +2 more

It is discovered that once a good basic ranking scheme is being used, the use of phrases does not have a major effect on precision at high ranks, and phrases are more useful at lower ranks where the connection between documents and relevance is more tenuous.

Support vector machines for text categorization

146 Citations2003Avik Basu, Clarence P. Walters +1 more

This paper compares artificial neural network and support vector machine algorithms for use as text classifiers of news items and identifies a reduction in feature set that provides improved results.

Journal of the American Society for Information ScienceThe effectiveness of a nonsyntactic approach to automatic phrase indexing for document retrieval

132 Citations1989Joel L. Fagan

It is not likely that phrase indexing of this kind will prove to be an important method of enhancing the performance of automatic document indexing and retrieval systems in operational environments, and a general syntactic analysis facility may be required.