Two-level hierarchical combination method for text classification
Expert Systems with ApplicationsPublished 5 August 2010
Wen Li, Duoqian Miao, Weili Wang
Citations29
SJR quartileQ1
SJR score1.85
SNIP2.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A two-level hierarchical algorithm that systematically combines the strength of support vector machine (SVM) and k nearest neighbor (KNN) techniques based on variable precision rough sets (VPRS) to improve the precision of text classification is proposed.
Abstract
Text classification has been recognized as one of the key techniques in organizing digital data. The intuition that each algorithm has its bias data and build a high performance classifier via some combination of different algorithm is a long motivation. In this paper, we proposed a two-level hierarchical algorithm that systematically combines the strength of support vector machine (SVM) and k nearest neighbor (KNN) techniques based on variable precision rough sets (VPRS) to improve the precision of text classification. First, an extension of regular SVM named variable precision rough SVM (VPRSVM), which partitions the feature space into three kinds of approximation regions, is presented. Second, a modified KNN algorithm named restrictive k nearest neighbor (RKNN) is put forward to reclassify texts in boundary region effectively and efficiently. The proposed algorithm overcomes the drawbacks of sensitive to noises of SVM and low efficiency of KNN. Experimental results compared with traditional algorithms indicate that the proposed method can improve the overall performance significantly.
Keywords
Computer Science
The Nature of Statistical Learning Theory
39,279 Citations1995Vladimir Vapnik
IEEE Transactions on Information TheoryNearest neighbor pattern classification
16,134 Citations1967Thomas M. Cover, Peter E. Hart
The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points, so it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.
IEEE Transactions on Evolutionary ComputationNo free lunch theorems for optimization
13,936 Citations1997David H. Wolpert, William G. Macready
A framework is developed to explore the connection between effective optimization algorithms and the problems they are solving and a number of "no free lunch" (NFL) theorems are presented which establish that for any algorithm, any elevated performance over one class of problems is offset by performance over another class.
International Journal of Parallel ProgrammingRough sets
12,138 Citations1982Zdzis aw Pawlak
The presented approach may be considered as an alternative to fuzzy sets theory and tolerance theory and some applications are outlined.
Lecture notes in computer scienceText categorization with Support Vector Machines: Learning with many relevant features
7,925 Citations1998Thorsten Joachims
SVMs achieve substantial improvements over the currently best performing methods and behave robustly over a variety of di-erent learning tasks, eliminating the need for manual parameter tuning.
A Comparative Study on Feature Selection in Text Categorization
4,766 Citations1997Yiming Yang, Jan Pedersen
DF thresholding, the simplest method with the lowest cost in computation, can be reliably used instead of IG or CHI when the computation of these measures are too expensive, and strong correlations between the DF, IG and CHI values of a term are found.
A re-examination of text categorization methods
2,660 Citations1999Yiming Yang, Xin Liu
The results show that SVM, kNN and LLSF signi cantly outperform NNet and NB when the number of positive training instances per category are small, and that all the methods perform comparably when the categories are over 300 instances.
Journal of Computer and System SciencesVariable precision rough set model
2,024 Citations1993Wojciech Ziarko
A generalized model of rough sets called variable precision model (VP-model), aimed at modelling classification problems involving uncertain or imprecise information, is presented and the main concepts are introduced formally and illustrated with simple examples.
In Defense of One-Vs-All Classification
1,387 Citations2004Ryan Rifkin, Aldebaro Klautau
It is argued that a simple "one-vs-all" scheme is as accurate as any other approach, assuming that the underlying binary classifiers are well-tuned regularized classifiers such as support vector machines.
Journal of Machine Learning ResearchIn Defense of One-Vs-All Classification
895 Citations2004RifkinRyan, KlautauAldebaro
Comparison of classifier methods: a case study in handwritten digit recognition
620 Citations2002Léon Bottou, Corinna Cortes +9 more
This paper compares the performance of several classifier algorithms on a standard database of handwritten digits by considering not only raw accuracy, but also training time, recognition time, and memory requirements.
Learning from Imbalanced Data Sets: A Comparison of Various Strategies *
453 Citations2000Nathalie Japkowicz
It is shown experimentally that, at least in the case of connectionist systems, class imbalances hinder the performance of standard classifiers and several approaches previously proposed to deal with the problem are compared.
Expert Systems with ApplicationsAn effective refinement strategy for KNN text classifier
275 Citations2005S TAN
This paper proposes a new refinement strategy, which is called as DragPushing, for the K-Nearest Neighbors Classifier, and shows that DragP pushing achieved a significant improvement on the performance of the KNN Classifier.
Evaluating text categorization
234 Citations1991David Lewis
Information SciencesRough set based 1-v-1 and 1-v-r approaches to support vector machine multi-classification
126 Citations2007Pawan Lingras, Cory J. Butz
This paper describes how binary classification with SVMs can be interpreted using rough sets and suggests two new approaches, extensions of 1-v-r and 1- v-1, to SVM multi-classification that allow for an error rate.
IEEE Transactions on Knowledge and Data EngineeringRough Cluster Quality Index Based on Decision Theory
114 Citations2009Pawan Lingras, Min Chen +1 more
The proposed cluster validity index based on the decision-theoretic rough set model by considering various loss functions is shown to help determine optimal number of clusters, as well as an important parameter called threshold in rough clustering.
A novel refinement approach for text categorization
106 Citations2005Songbo Tan, Xueqi Cheng +3 more
The results indicate that in each case, the refined classifiers achieve significant performance improvement over the base classifiers used, and the performance of the Refined Centroid Classifier implemented is comparable, if not better, to that of state-of-the-art support vector machine (SVM)-based classifier, but offers a much lower computational cost.
Information SciencesA rough margin based support vector machine
81 Citations2008Jianyun Zhang, Yinghong Wang
By introducing the rough set theory into the support vector machine (SVM), a rough margin based SVM (RMSVM) is proposed to deal with the overfitting problem due to outliers.
OmegaPrecision parameter in the variable precision rough sets model: an application
80 Citations2004Chao‐Ton Su, Jyh-Hwa Hsu
This study proposes an effective method to find the [beta]-reducts and compares the implementation results from the proposed method with the neural network approach, showing a better performance.
Expert Systems with ApplicationsRough set based hybrid algorithm for text classification
76 Citations2008Duoqian Miao, Qiguo Duan +2 more
A hybrid algorithm based on variable precision rough set is proposed to combine the strength of both k-nearest neighbor and Rocchio techniques and overcome their weaknesses and indicates that the novel algorithm achieves significant performance improvement.
NeurocomputingResearch of reduct features in the variable precision rough set model
60 Citations2008Jiayang Wang, Jie Zhou
Some kinds of reduction anomalies are analyzed in detail, the concept of inclusion degree (@b) threshold is put forward and the relationship between inclusion degree and classification quality is discussed, which develops the variable precision rough set mode further.
Information RetrievalThe Combination of Text Classifiers Using Reliability Indicators
59 Citations2004Paul N. Bennett, Susan Dumais +1 more
A probabilistic method for combining classifiers that considers the context-sensitive reliabilities of contributing classifiers and harnesses reliability indicators—variables that provide signals about the performance of classifiers in different situations.
IEICE Transactions on Information and SystemsImproved Classification for Problem Involving Overlapping Patterns
37 Citations2007Yansong Tang, Jing Gao
Simulation results indicate that more reasonable and accurate information about the pattern's category could be obtained by use of DR-SVM, a novel multi-model classifier which combines SVM classifier with kNN algorithm under rough set technique.
Fundamenta InformaticaeRough set Based Ensemble Classifier forWeb Page Classification
35 Citations2007Suman Saha, C.A. Murthy +1 more
A rough set based meta classifier is introduced to classify web pages that improves accuracy of classification uniformly over some benchmark corpora and beats other ensemble approaches in accuracy by a decisive margin, thus demonstrating the theoretical results.
Lecture notes in computer scienceSV-kNNC: An Algorithm for Improving the Efficiency of k-Nearest Neighbor
21 Citations2006Anantaporn Srisawat, Tanasanee Phienthrakul +1 more
2007 IEEE International Conference on Granular Computing (GRC 2007)Fuzzy Vector Quantization for Network Intrusion Detection
17 Citations2007Dat Tran, Wanli Ma +2 more
Experiments performed on the KDD CUP 1999 dataset show that time based traffic features in the last two second time window should be selected to obtain highest detection rates.
A model for handling approximate, noisy or incomplete labeling in text classification
15 Citations2005Ganesh Ramakrishnan, Krishna Prasad Chitrapura +2 more
A Bayesian model, BayesANIL, that is capable of estimating uncertainties associated with the labeling process, and provides an intuitive modification to the EM iterations by re-estimating the empirical.
Lecture notes in computer scienceAn Examination of Feature Selection Frameworks in Text Categorization
14 Citations2005Bong Chih How, Wong Ting Kiong
This paper explored the different frameworks of feature selection to the task of text categorization on the Reuters(10) and Reuters(115) datasets (variants of Reuters-21578 corpus) and investigated the efficiency of 7 different local or global feature selections corresponds the use of local and universal dictionary.
International Journal of Intelligent SystemsThe introduction and utilization of (l,u)-graphs in the extended variable precision rough sets model
10 Citations2003Malcolm J. Beynon
The notion of the ( l, u)‐graph is introduced, which elucidates the effect of the choice of the l and u values on the associated levels of (l, U)‐DoD and (l‐U)‐QoC in the extended variable precision rough sets model.
Two-Level Hierarchical Hybrid SVM-RVM Classification Model
10 Citations2006Catarina Silva, Bernardete Ribeiro
A two-level hierarchical hybrid SVM-RVM model is proposed to combine the best of both learning machines and shows the benefits of the hierarchical approach on a text classification task, where the two-levels outperform both learning Machines.
2007 IEEE International Conference on Granular Computing (GRC 2007)Precision and Recall in Rough Support Vector Machines
9 Citations2007Li Lu, T. Lin
This paper shows how to extend the use of precision and recall from a SVM implementation to a RSVM implementation, and demonstrates in practice with the help of Gist, a popular S VM implementation.
Lecture notes in computer scienceThe Elucidation of an Iterative Procedure to ß-Reduct Selection in the Variable Precision Rough Sets Model
5 Citations2004Malcolm James Beynon
The role of an iterative procedure in the necessary s-reduct selection process is investigated for the variable precision rough sets model (VPRS) and a s value defining the VPRS related majority inclusion relation to object classification is defined.