Semi-supervised learning of mixture models
Published 21 August 2003
Fábio Gagliardi Cozman, Ira L. Cohen, Marcelo César Cirelo
Citations192
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper analyzes the performance of semi-supervised learning of mixture models and shows that unlabeled data can lead to an increase in classification error even in situations where additional labeled data would decrease classification error.
Abstract
This paper analyzes the performance of semisupervised learning of mixture models. We show that unlabeled data can lead to an increase in classification error even in situations where additional labeled data would decrease classification error. We present a mathematical analysis of this “degradation ” phenomenon and show that it is due to the fact that bias may be adversely affected by unlabeled data. We discuss the impact of these theoretical results to practical situations. 1.
Keywords
Computer Science
Machine LearningBayesian Network Classifiers
4,738 Citations1997Nir Friedman, Dan Geiger +1 more
Tree Augmented Naive Bayes (TAN) is single out, which outperforms naive Bayes, yet at the same time maintains the computational simplicity and robustness that characterize naive Baye.
Stochastic modelling and applied probabilityA Probabilistic Theory of Pattern Recognition
3,277 Citations1996Luc Devroye, László Györfi +1 more
The Bayes Error and Vapnik-Chervonenkis theory are applied as guide for empirical classifier selection on the basis of explicit specification and explicit enforcement of the maximum likelihood principle.
Machine LearningText Classification from Labeled and Unlabeled Documents using EM
2,749 Citations2000Kamal Nigam, Andrew Kachites McCallum +2 more
This paper shows that the accuracy of learned text classifiers can be improved by augmenting a small number of labeled training documents with a large pool of unlabeled documents, and presents two extensions to the algorithm that improve classification accuracy under these conditions.
Journal of the American Statistical AssociationDiscriminant Analysis and Statistical Pattern Recognition.
2,671 Citations1993Douglas M. Hawkins, Geoffrey J. McLachlan
Queensland's institutional digital repository (The University of Queensland)Mixture models : inference and applications to clustering
2,074 Citations1988Geoffrey J. McLachlan, K. E. Basford
Unsupervised Models for Named Entity Classification
817 Citations1999Michael Collins, Yoram Singer
It is shown that the use of unlabeled data can reduce the requirements for supervision to just 7 simple "seed" rules, gaining leverage from natural redundancy in the data.
Employing EM and Pool-Based Active Learning for Text Classification
681 Citations1998Andrew McCallum, Kamal Nigam
The Query-by-Committee method of active learning is modified to use the unlabeled pool for explicitly estimating document density when selecting examples for labeling, and the combination of EM and active learning requires only slightly more than half as many labeled training examples to achieve the same accuracy as either the improved active learning or EM alone.
IEEE Transactions on Geoscience and Remote SensingThe effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon
572 Citations1994B.M. Shahshahani, D. A. Landgrebe
By using additional unlabeled samples that are available at no extra cost, the performance may be improved, and therefore the Hughes phenomenon can be mitigated and therefore more representative estimates can be obtained.
Encyclopedia of Machine Learning and Data MiningLearning from Labeled and Unlabeled Data
469 Citations2017Claude Sammut, Geoffrey I. Webb
Enhancing Supervised Learning with Unlabeled Data
459 Citations2000Sally A. Goldman, Yan Zhou
A new semi-supervised learning method called co-learning that is designed to use unlabeled data to enhance standard supervised learning algorithms to leverage off the fact that they have different representations of the hypotheses and are likely to detect different patterns in labeled data.
The Annals of Mathematical StatisticsLimiting Behavior of Posterior Distributions when the Model is Incorrect
402 Citations1966Robert H. Berk
Neural Information Processing SystemsA Mixture of Experts Classifier with Learning Based on Both Labelled and Unlabelled Data
289 Citations1996David J. Miller, Hasan S. Uyar
A classifier structure and learning algorithm that make effective use of unlabelled data to improve performance and is a "mixture of experts" structure that is equivalent to the radial basis function (RBF) classifier, but unlike RBFs, is amenable to likelihood-based training.
Machine LearningMaximum Entropy Discrimination
259 Citations2004Tony Jebara
A general framework for discriminative estimation based on the maximum entropy principle and its extensions is presented and preliminary experimental results are indicative of the potential in these techniques.
IEEE Transactions on Information TheoryThe relative value of labeled and unlabeled samples in pattern recognition with an unknown mixing parameter
206 Citations1996V. Castelli, Thomas M. Cover
It is shown that labeled samples are necessary to construct a classification rule and that they are exponentially more valuable than unlabeled samples.
Pattern Recognition LettersOn the exponential value of labeled samples
201 Citations1995Vittorio Castelli, Thomas M. Cover
The first labeled sample reduces the risk from 1 2 to 2R ∗ (1−R∗ ) and subsequent labeled samples in the training set reduce the probability of error exponentially fast to the Bayes risk.
Learning Bayesian network classifiers for facial expression recognition both labeled and unlabeled data
156 Citations2003Ira L. Cohen, Nicu Sebe +3 more
A classification driven stochastic structure search algorithm for learning the structure of Bayesian network classifiers is introduced, and it is shown that with moderate size labeled training sets and large amount of unlabeled data, the method can utilize unlabeling data to improve classification performance.
Journal of the American Statistical AssociationNormal Discrimination with Unclassified Observations
144 Citations1978Terence J. O’Neill
Unlabeled Data Can Degrade Classification Performance of Generative Classifiers
130 Citations2002Fábio Gagliardi Cozman, Ira L. Cohen
It is shown that unlabeled data can degrade the performance of a classifier when there are discrepancies between modeling assumptions used to build the classifier and the actual model that generates the data.
Munich Personal RePEc Archive (Ludwig Maximilian University of Munich)Using unlabeled data to improve text classification
129 Citations2001Kanal Paul Nigam, Tom M. Mitchell
This dissertation demonstrates that supervised learning algorithms that use a small number of labeled examples and many inexpensive unlabeled examples can create high-accuracy text classifiers.
Learning from a mixture of labeled and unlabeled examples with parametric side information
97 Citations1995Joel Ratsaby, Santosh S. Venkatesh
The tradeoff between labeled and unlabeled sample complexities in learning is investigated and pendent sampling from the distribution of pairs (x, y) is shown.
Lecture notes in computer sciencePositive and Unlabeled Examples Help Learning
94 Citations1999Francesco de Comité, François Denis +2 more
This work addresses the problem of learning with the help of positive and unlabeled data given a small number of labeled examples and presents both theoretical and empirical arguments showing that learning algorithms can be improved by the use of both unlabeling and positive data.
Probabilistic Modeling for Face Orientation Discrimination: Learning from Labeled and Unlabeled Data
87 Citations1998Shumeet Baluja
BiometrikaThe efficiency of a linear discriminant function based on unclassified initial samples
77 Citations1978S. Gànesalingam, Geoffrey J. McLachlan
Robustness in Statistical Pattern Recognition
70 Citations1996Yu. S. Kharin
This paper presents Cluster Analysis under Distorted Model Assumptions and Basic Notions in Decision Rule Robustness, a study of decision rule robustness in probability models of data and Optimal Decision Rules.
Combining labeled and unlabeled data for text classification with a large number of categories
63 Citations2002Rayid Ghani
This work develops a framework to incorporate unlabeled data in the error-correcting output coding (ECOC) setup by decomposing multiclass problems into multiple binary problems and then using co-training to learn the individual binary classification problems.
Continuation methods for mixing heterogeneous sources
31 Citations2002Adrian Corduneanu, Tommi Jaakkola
IEEE Transactions on ComputersOn the Asymptotic Improvement in the Out- come of Supervised Learning Provided by Additional Nonsupervised Learning
26 Citations1970David B. Cooper, John H. Freeman
Simple mathematical expressions are derived for the improvement in supervised learning provided by additional nonsupervised learning when the number of learning samples is large so that asymptotic approximations are appropriate.
The relative value of labeled and unlabeled samples in pattern recognition
24 Citations2005V. Castelli, Thomas M. Cover
It is argued that labeled samples are exponentially more valuable than unlabeled samples and identify the (exponent as the Bhatthacharyya distance).
Purdue e-Pubs (Purdue University)Classification of multispectral data by joint supervised-unsupervised learning
18 Citations1993B.M. Shahshahani, D. A. Landgrebe
It is shown that by adding the unlabeled samples to the classifier design process, better estimates for the discriminant functions can be obtained and the peaking phenomenon that is observed in the performance versus dimensionality curves, can be mitigated.