Feature selection for text classification with Naïve Bayes
Expert Systems with ApplicationsPublished 25 June 2008
Jingnian Chen, Houkuan Huang, Shengfeng Tian, Youli Qu
Citations572
SJR quartileQ1
SJR score1.85
SNIP2.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Two feature evaluation metrics for the Naive Bayesian classifier applied on multi-class text datasets are presented: Multi-class Odds Ratio (MOR), and Class Discriminating Measure (CDM).
Abstract
As an important preprocessing technology in text classification, feature selection can improve the scalability, efficiency and accuracy of a text classifier. In general, a good feature selection method should consider domain and algorithm characteristics. As the Naïve Bayesian classifier is very simple and efficient and highly sensitive to feature selection, so the research of feature selection specially for it is significant. This paper presents two feature evaluation metrics for the Naïve Bayesian classifier applied on multi-class text datasets: Multi-class Odds Ratio (MOR), and Class Discriminating Measure (CDM). Experiments of text classification with Naïve Bayesian classifiers were carried out on two multi-class texts collections. As the results indicate, CDM and MOR gain obviously better selecting effect than other feature selection approaches.
Keywords
Computer Science
IEEE Transactions on Information TheoryNearest neighbor pattern classification
16,134 Citations1967Thomas M. Cover, Peter E. Hart
The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points, so it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.
Lecture notes in computer scienceText categorization with Support Vector Machines: Learning with many relevant features
7,925 Citations1998Thorsten Joachims
SVMs achieve substantial improvements over the currently best performing methods and behave robustly over a variety of di-erent learning tasks, eliminating the need for manual parameter tuning.
Proceedings of the 25th international conference on Machine learning
5,542 Citations2008William W. Cohen, Andrew McCallum +1 more
A Comparative Study on Feature Selection in Text Categorization
4,766 Citations1997Yiming Yang, Jan Pedersen
DF thresholding, the simplest method with the lowest cost in computation, can be reliably used instead of IG or CHI when the computation of these measures are too expensive, and strong correlations between the DF, IG and CHI values of a term are found.
A comparison of event models for naive bayes text classification
3,224 Citations1998Andrew McCallum, Kamal Nigam
It is found that the multi-variate Bernoulli performs well with small vocabulary sizes, but that the multinomial performs usually performs even better at larger vocabulary sizes--providing on average a 27% reduction in error over the multi -variateBernoulli model at any vocabulary size.
An extensive empirical study of feature selection metrics for text classification
2,389 Citations2003George Forman
Elsevier eBooksIrrelevant Features and the Subset Selection Problem
2,374 Citations1994George H. John, Ron Kohavi +1 more
A method for feature subset selection using cross-validation that is applicable to any induction algorithm is described, and experiments conducted with ID3 and C4.5 on artificial and real datasets are discussed.
Lecture notes in computer scienceNaive (Bayes) at forty: The independence assumption in information retrieval
2,092 Citations1998David Lewis
The naive Bayes classifier, currently experiencing a renaissance in machine learning, has long been a core technique in information retrieval, and some of the variations used for text retrieval and classification are reviewed.
Information RetrievalAn Evaluation of Statistical Approaches to Text Categorization
1,946 Citations1999Yiming Yang
Analysis and empirical evidence suggest that the evaluation results on some versions of Reuters were significantly affected by the inclusion of a large portion of unlabelled documents, mading those results difficult to interpret and leading to considerable confusions in the literature.
IEEE Transactions on Knowledge and Data EngineeringSome Effective Techniques for Naive Bayes Text Classification
572 Citations2006Sang‐Bum Kim, Kyoung-Soo Han +2 more
Feature Selection for Unbalanced Class Distribution and Naive Bayes
432 Citations1999Dunja Mladenić, Marko Grobelnik
This paper describes an approach to feature subset selection that takes into account problem speciics and learning algorithm characteristics, and shows that considering domain and algorithm characteristics signiicantly improves the results of classiication.
ACM Transactions on Information SystemsAn example-based mapping method for text categorization and retrieval
405 Citations1994Yiming Yang, Christopher G. Chute
It is evident that the LLSF approach uses the relevance information effectively within human decisions of categorization and retrieval, and achieves a semantic mapping of free texts to their representations in an indexing language.
Expert Systems with ApplicationsNeighbor-weighted K-nearest neighbor for unbalanced text corpus
344 Citations2005S TAN
The neighbor-weighted K-nearest neighbor algorithm, i.e. NWKNN, is proposed, which achieves significant classification performance improvement on imbalanced corpora.
Expert Systems with ApplicationsA novel feature selection algorithm for text categorization
338 Citations2006Wenqian Shang, Houkuan Huang +4 more
This study designs a novel Gini index algorithm to reduce the high dimensionality of the feature space and builds a new measure function of Gini Index constructed and made to fit text categorization.
Lecture notes in computer scienceNaive Bayes for Text Classification with Unbalanced Classes
236 Citations2006Eibe Frank, Remco Bouckaert
This paper proposes an appropriate correction by adjusting attribute priors that can significantly improve the area under the ROC curve and shows that the modified version of MNB is very closely related to the simple centroid-based classifier and compare the two methods empirically.
Decision Support SystemsFeature selection on hierarchy of web documents
129 Citations2002Dunja Mladenić, Marko Grobelnik
This work proposes a way of dividing the problem into subproblems, each representing one of the categories included in the Yahoo hierarchy, and proposes a set of independent classifiers, each used to predict probability that a new example is a member of the corresponding category.
Zhongwen xinxi xuebaoStudy on Feature Selection in Chinese Text Categorization
23 Citations2004Ming Zhao
Multi Class Odds Ratio, a variant of Odds ratio which is often used in binary classification, and a new feature selection method based on Class Discriminating Words are proposed and combined with the classic VSM classifier based on cosine similarity and the Nave Bayes classifier gain the best selecting effect.
Computer Knowledge and TechnologyA Study on Feature Selection in Chinese Text Categorization
16 Citations2007LV Zhi-long
A furthermore experiment proved that the combined feature selection method of a combined type is effective and suppress effectively the lack of the high or low frequency word.