Some Effective Techniques for Naive Bayes Text Classification
IEEE Transactions on Knowledge and Data EngineeringPublished 29 September 2006
Sang‐Bum Kim, Kyoung-Soo Han, Hae‐Chang Rim, Sung Hyon Myaeng
Citations572
SJR quartileQ1
SJR score2.57
SNIP3.30
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
While naive Bayes is quite effective in various data mining tasks, it shows a disappointing result in the automatic text classification problem. Based on the observation of naive Bayes for the natural language text, we found a serious problem in the parameter estimation process, which causes poor results in text classification domain. In this paper, we propose two empirical heuristics: per-document text normalization and feature weighting method. While these are somewhat ad hoc methods, our proposed naive Bayes text classifier performs very well in the standard benchmark collections, competing with state-of-the-art text classifiers based on a highly complex learning method such as SVM.
Keywords
Computer Science
Lecture notes in computer scienceText categorization with Support Vector Machines: Learning with many relevant features
7,925 Citations1998Thorsten Joachims
SVMs achieve substantial improvements over the currently best performing methods and behave robustly over a variety of di-erent learning tasks, eliminating the need for manual parameter tuning.
A Comparative Study on Feature Selection in Text Categorization
4,766 Citations1997Yiming Yang, Jan Pedersen
DF thresholding, the simplest method with the lowest cost in computation, can be reliably used instead of IG or CHI when the computation of these measures are too expensive, and strong correlations between the DF, IG and CHI values of a term are found.
JAMAWhat's the Relative Risk?
3,993 Citations1998Jun Zhang, Kai Yu
This work proposes a simple method to approximate a risk ratio from the adjusted odds ratio and derive an estimate of an association or treatment effect that better represents the true relative risk.
Machine LearningOn the Optimality of the Simple Bayesian Classifier under Zero-One Loss
3,072 Citations1997Pedro Domingos, Michael J. Pazzani
The Bayesian classifier is shown to be optimal for learning conjunctions and disjunctions, even though they violate the independence assumption, and will often outperform more powerful classifiers for common training set sizes and numbers of attributes, even if its bias is a priori much less appropriate to the domain.
Machine LearningText Classification from Labeled and Unlabeled Documents using EM
2,749 Citations2000Kamal Nigam, Andrew Kachites McCallum +2 more
This paper shows that the accuracy of learned text classifiers can be improved by augmenting a small number of labeled training documents with a large pool of unlabeled documents, and presents two extensions to the algorithm that improve classification accuracy under these conditions.
A re-examination of text categorization methods
2,660 Citations1999Yiming Yang, Xin Liu
The results show that SVM, kNN and LLSF signi cantly outperform NNet and NB when the number of positive training instances per category are small, and that all the methods perform comparably when the categories are over 300 instances.
Machine LearningBoosTexter: A Boosting-based System for Text Categorization
2,194 Citations2000Robert E. Schapire, Yoram Singer
This work describes in detail an implementation, called BoosTexter, of the new boosting algorithms for text categorization tasks, and presents results comparing the performance of Boos Texter and a number of other text-categorization algorithms on a variety of tasks.
Lecture notes in computer scienceNaive (Bayes) at forty: The independence assumption in information retrieval
2,092 Citations1998David Lewis
The naive Bayes classifier, currently experiencing a renaissance in machine learning, has long been a core technique in information retrieval, and some of the variations used for text retrieval and classification are reviewed.
Inductive learning algorithms and representations for text categorization
1,465 Citations1998Susan Dumais, John Platt +2 more
A comparison of the effectiveness of five different automatic learning algorithms for text categorization in terms of learning speed, realtime classification speed, and classification accuracy is compared.
Toward optimal feature selection
1,444 Citations1996Daphne Koller, Mehran Sahami
An efficient algorithm for feature selection which computes an approximation to the optimal feature selection criterion is given, showing that the algorithm effectively handles datasets with a very large number of features.
A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization
1,264 Citations1997Thorsten Joachims
A Probabilistic analysis of the Rocchio relevance feedback algorithm, one of the most popular learning methods from information retrieval, is presented in a text categorization framework and suggests that the probabilistic algorithms are preferable to the heuristic Rocchio classifier.
Employing EM and Pool-Based Active Learning for Text Classification
681 Citations1998Andrew McCallum, Kamal Nigam
The Query-by-Committee method of active learning is modified to use the unlabeled pool for explicitly estimating document density when selecting examples for labeling, and the combination of EM and active learning requires only slightly more than half as many labeled training examples to achieve the same accuracy as either the improved active learning or EM alone.
ACM Transactions on Information SystemsAn example-based mapping method for text categorization and retrieval
405 Citations1994Yiming Yang, Christopher G. Chute
It is evident that the LLSF approach uses the relevance information effectively within human decisions of categorization and retrieval, and achieves a semantic mapping of free texts to their representations in an indexing language.
Scholarworks (University of Massachusetts Amherst)Representation and Learning in Information Retrieval
361 Citations1991David Lewis
A new theoretical model for text classification systems, including systems for document retrieval, automated indexing, electronic mail filtering, and similar tasks, is introduced, suggesting that the poor statistical characteristics of a syntactic indexing phrase representation negate its desirable semantic characteristics.
Text REtrieval ConferenceOkapi at TREC-7: Automatic Ad Hoc, Filtering, VLC and Interactive.
347 Citations1998Stephen Robertson, Steve Walker +1 more
Three runs were submitted: medium (title and description), short (title only) and a run which was a combination of a long run (title, description and narrative) with the medium and short runs, which led to the discovery that due to a mistake in the indexing procedures part of the LA Times documents had been indexed.
Lecture notes in computer scienceFeature subset selection in text-learning
171 Citations1998Dunja Mladenić
Experimental comparison given on real-world data collected from Web users shows that characteristics of the problem domain and machine learning algorithm should be considered when feature scoring measure is selected.
An Empirical Study of Feature Selection for Text Categorization based on Term Weightage
54 Citations2004Chih How Bong, K. Lakshmi Narayanan
A local feature selection measure namely, Categorical Descriptor Term (CTD), derived based on classic term weighting scheme, TFIDF, which can perform comparatively well with other FS measures, especially on collection with highly overlapped topics.
IEEE/WIC/ACM International Conference on Web Intelligence (WI'04)An Empirical Study of Feature Selection for Text Categorization based on Term Weightage
18 Citations2005Bong Chih How, K. Narayanan