Co-training with relevant random subspaces
NeurocomputingPublished 15 March 2010
Yusuf Yaslan, Zehra Çataltepe
Citations94
SJR quartileQ1
SJR score1.47
SNIP1.94
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Experiments show that the relevant random subspace Co-training (Rel-RASCO) algorithm outperforms both RASCO and Co- Training in terms of the accuracy achieved at the end of Co- training.
Abstract
We introduce the relevant random subspace Co-training (Rel-RASCO) algorithm which produces relevant random subspaces and then does semi-supervised ensemble learning using those subspaces and unlabeled data. Ensemble learning algorithms may benefit from diversity of classifiers used. However, for high dimensional data choosing subspaces randomly, as in RASCO (Random Subspace Method for Co-training, Wang et al. 2008 [5]) algorithm, may produce diverse but inaccurate classifiers. We produce relevant random subspaces by means of drawing features with probabilities proportional to their relevances measured by the mutual information between features and class labels. We show that Rel-RASCO achieves better accuracy by this relevant and random subspace selection scheme. Experiments on five real and one synthetic datasets show that Rel-RASCO algorithm outperforms both RASCO and Co-training in terms of the accuracy achieved at the end of Co-training.
Keywords
Computer Science
Journal of the Royal Statistical Society Series B (Statistical Methodology)Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
49,657 Citations1977A. P. Dempster, N. M. Laird +1 more
Machine LearningInduction of Decision Trees
14,815 Citations1986J. R. Quinlan
This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail, which is described in detail.
Machine LearningInduction of decision trees
12,383 Citations1986J. R. Quinlan
This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail, which is described in detail.
IEEE Transactions on Pattern Analysis and Machine IntelligenceFeature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy
10,470 Citations2005Hanchuan Peng, Fuhui Long +1 more
This work derives an equivalent form, called minimal-redundancy-maximal-relevance criterion (mRMR), for first-order incremental feature selection, and presents a two-stage feature selection algorithm by combining mRMR and other more sophisticated feature selectors (e.g., wrappers).
Advances in industrial controlPattern Classification
7,884 Citations2000Evan L. Russell, Leo H. Chiang +1 more
IEEE Transactions on Pattern Analysis and Machine IntelligenceThe random subspace method for constructing decision forests
6,737 Citations1998Tin Kam Ho
A method to construct a decision tree based classifier is proposed that maintains highest accuracy on training data and improves on generalization accuracy as it grows in complexity.
Combining labeled and unlabeled data with co-training
5,604 Citations1998Avrim Blum, Tom M. Mitchell
The MIT Press eBooksSemi-Supervised Learning
4,308 Citations2006Olivier Chapelle, Schölkopf, B. +1 more
This first comprehensive overview of semi-supervised learning presents state-of-the-art algorithms, a taxonomy of the field, selected applications, benchmark experiments, and perspectives on ongoing and future research.
TechnometricsCombining Pattern Classifiers: Methods and Algorithms
3,234 Citations2005Subhash Bagui
This chapter discusses the development of the Spatial Point Pattern Analysis Code in S–PLUS, which was developed in 1993 by P. J. Diggle and D. C. Griffith.
IEEE Transactions on Speech and Audio ProcessingMusical genre classification of audio signals
2,777 Citations2002George Tzanetakis, Patrick Cook
The automatic classification of audio signals into an hierarchy of musical genres is explored and three feature sets for representing timbral texture, rhythmic content and pitch content are proposed.
Foundations of genetic algorithmsA Comparative Analysis of Selection Schemes Used in Genetic Algorithms
2,400 Citations1991David E. Goldberg, Kalyanmoy Deb
A number of selection schemes commonly used in modern genetic algorithms are compared on the basis of solutions to deterministic difference or differential equations, verified through computer simulations to provide convenient approximate or exact solutions and useful convergence time and growth ratio estimates.
IEEE Transactions on Knowledge and Data EngineeringTri-training: exploiting unlabeled data using three classifiers
1,185 Citations2005Zhi‐Hua Zhou, Ming Li
Experiments on UCI data sets and application to the Web page classification task indicate that tri-training can effectively exploit unlabeled data to enhance the learning performance.
Analyzing the effectiveness and applicability of co-training
1,028 Citations2000Kamal Nigam, Rayid Ghani
It is demonstrated that when learning from labeled and unlabeled data, algorithms explicitly leveraging a natural independent split of the features outperform algorithms that do not and may out-perform algorithms not using a split.
IEEE Transactions on Pattern Analysis and Machine IntelligenceAsymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval
876 Citations2006Dacheng Tao, Xiaoou Tang +2 more
An asymmetric bagging and random subspace SVM (ABRS-SVM) is built to solve three problems and further improve the relevance feedback performance.
Research Commons (University of Waikato)Weka: Practical machine learning tools and techniques with Java implementations
634 Citations1999Ian H. Witten, Eibe Frank +4 more
The Waikato Environment for Knowledge Analysis (Weka) is a comprehensive suite of Java class libraries that implement many state-of-the-art machine learning and data mining algorithms.
Minimum redundancy feature selection from microarray gene expression data
498 Citations2004C. Ding, Hujin Peng
Feature sets obtained through the minimum redundancy - maximum relevance framework represent broader spectrum of characteristics of phenotypes than those obtained through standard ranking methods; they are more robust, generalize well to unseen data, and lead to significantly improved classifications in extensive experiments on 5 gene expressions data sets.
Encyclopedia of Machine Learning and Data MiningLearning from Labeled and Unlabeled Data
469 Citations2017Claude Sammut, Geoffrey I. Webb
Enhancing Supervised Learning with Unlabeled Data
459 Citations2000Sally A. Goldman, Yan Zhou
A new semi-supervised learning method called co-learning that is designed to use unlabeled data to enhance standard supervised learning algorithms to leverage off the fact that they have different representations of the hypotheses and are likely to detect different patterns in labeled data.
IEEE Transactions on Systems Man and Cybernetics - Part A Systems and HumansImprove Computer-Aided Diagnosis With Machine Learning Techniques Using Undiagnosed Samples
381 Citations2007Ming Li, Zhi‐Hua Zhou
Case studies on three medical data sets and a successful application to microcalcification detection for breast cancer diagnosis show that undiagnosed samples are helpful in building CAD systems, and Co-Forest is able to enhance the performance of the hypothesis that is learned on only a small amount of diagnosed samples by utilizing the available undiognosed samples.
Decision Support SystemsPartitioning-based clustering for Web document categorization
261 Citations1999Daniel Boley, Maria Gini +7 more
Two new clustering algorithms are introduced that can effectively cluster documents, even in the presence of a very high dimensional feature space, and do not require pre-specified ad hoc distance functions and are capable of automatically discovering document similarities or associations.
Bayesian Factor Regression Models in the “Large p, Small n” Paradigm
212 Citations2003Mike West
Lecture notes in computer scienceDiversity versus Quality in Classification Ensembles Based on Feature Selection
206 Citations2000Pádraig Cunningham, John G. Carney
It is argued that feature subset-selection has emerged as a useful technique for creating diversity in ensembles and this diversity needs to be monitored in the creation of the ensemble, and an entropy measure of the outputs of the Ensemble Diversity Index is proposed.
Democratic co-learning
201 Citations2005Yaojian Zhou, Shelley Goldman
This work presents democratic colearning in which multiple algorithms instead of multiple views enable learners to label data for each other, a new example selection method for active learning.
Email classification with co-training
191 Citations2011Svetlana Kiritchenko, Stan Matwin
The main problems in text classification are lack of labeled data, as well as the cost of labeling the unlabeled data, which are addressed by exploring co-training - an algorithm that uses unlabeling data along with a few labeled examples to boost the performance of a classifier.
IEEE Transactions on Image ProcessingMultitraining Support Vector Machine for Image Retrieval
142 Citations2006Jing Li, N.M. Allinson +2 more
A new machine learning technique is developed, multitraining SVM (MTSVM), which combines the merits of the cotraining technique and a random sampling method in the feature space that consistently improves the performance over conventional SVM-based RFs in terms of precision and standard deviation.
TMG: A MATLAB Toolbox for Generating Term-Document Matrices from Text Collections
135 Citations2006Dimitrios Zeimpekis, Efstratios Gallopoulos
This work presents TMG, a research and teaching toolbox for the generation of sparse tdms from text collections and for the incremental modification of these tdms by means of additions or deletions, written entirely in MATLAB.
Co-training for predicting emotions with spoken dialogue data
88 Citations2004Beatriz Maeireizo, Diane Litman +1 more
This paper investigates the applicability of Co-training to train classifiers that predict emotions in spoken dialogues and applies the wrapper approach with Forward Selection and Naive Bayes, to reduce the dimensionality of the feature set.
Pattern Analysis and ApplicationsInput decimated ensembles
77 Citations2003Kagan Tumer, Nikunj C. Oza
Input Decimation (ID) is explored, a method which selects feature subsets for their ability to discriminate among the classes and uses these subsets to decouple the base classifiers, and results indicate that input decimated ensembles outperform ensemble whose base classifier use all the input features.
IEEE Transactions on Audio Speech and Language ProcessingModeling timbre distance with temporal statistics from polyphonic music
67 Citations2005Fabian Mörchen, Alfred Ultsch +2 more
A random subspace method for co-training
62 Citations2008Jiao Wang, Luo Siwei +1 more
A novel co-training style algorithm, RASCO (for RAndom Subspace CO-training), is proposed which uses stochastic discrimination theory to extend co- Training to multi-view situation and demonstrates that RASco is more effective than other co- training style algorithms.
Co-training by Committee: A New Semi-supervised Learning Framework
56 Citations2008Mohamed Farouk Abdel Hady, Friedhelm Schwenker
Experimental studies on CoBC using bagging, AdaBoost and the random subspace method as ensemble learners demonstrate that error diversity among classifiers leads to an effective co-training that requires neither redundant and independent views nor different learning algorithms.
Lecture notes in computer scienceSemi-supervised Multiple Classifier Systems: Background and Research Directions
34 Citations2005Fabio Roli
The goal of this paper is to review the background results that can be exploited to promote research on semi-supervised multiple classifier systems, and to outline some future research directions.
Lecture notes in computer scienceUsing Co-training and Self-training in Semi-supervised Multiple Classifier Systems
34 Citations2006Luca Didaci, Fabio Roli
Results on benchmarking data sets show that co-training and self-training allow exploiting unlabelled data in different types of multiple classifiers systems.
The 2006 IEEE International Joint Conference on Neural Network ProceedingsCo-training using RBF Nets and Different Feature Splits
27 Citations2006Felix Feger, Irena Koprinska
A new graph-based feature splitting algorithm maxlnd is proposed, which creates a balanced split maximizing the independence between the two feature sets, which shows that RBF net is successful in a co-training setting, outperforming SVM and NB.
Semi-Supervised Classification of Musical Genre using Multi-View Features.
10 Citations2005Yungpeng Xu, Changshui Zang +1 more
This work adopts three most popular feature sets to form two feature views for classification, so as to extract as much discriminative information contained by different feature sets as possible.
Lecture notes in computer scienceRandom Relevant and Non-redundant Feature Subspaces for Co-training
2 Citations2009Yusuf Yaslan, Zehra Çataltepe
Two algorithms for selecting relevant and non-redundant feature subspaces for Co-training are introduced and it is demonstrated that the proposed algorithms outperform both RASCO and Co- Training in terms of accuracy achieved at the end of Co- training.