Using Unlabeled Data for Supervised Learning

Neural Information Processing SystemsPublished 27 November 1995

Geoffrey G. Towell

Citations11

TL;DR

Empirical tests show that the technique described in this paper can significantly improve the accuracy of a supervised learner when the learner is well below its asymptotic accuracy level.

Abstract

Many classification problems have the property that the only costly part of obtaining examples is the class label. This paper suggests a simple method for using distribution information contained in unlabeled examples to augment labeled examples in a supervised training framework. Empirical tests show that the technique described in this paper can significantly improve the accuracy of a supervised learner when the learner is well below its asymptotic accuracy level.

Keywords

Computer Science

Elsevier eBooksHeterogeneous Uncertainty Sampling for Supervised Learning

1,142 Citations1994David Lewis, Jason Catlett

This work test the use of one classifier (a highly efficient probabilistic one) to select examples for training another (the C4.5 rule induction program) and finds that the uncertainty samples yielded classifiers with lower error rates than random samples ten times larger.

Artificial IntelligenceKnowledge-based artificial neural networks

739 Citations1994Geoffrey G. Towell, Jude Shavlik

These tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several techniques proposed by biologists.

Supervised learning from incomplete data via an EM approach

543 Citations1993Zoubin Ghahramani, Michael I. Jordan

A framework based on maximum likelihood density estimation for learning from high-dimensional data sets with arbitrary patterns of missing data is presented and results from a classification benchmark--the iris data set--are presented.

Neural Network Perception for Mobile Robot Guidance

481 Citations1993Dean Pomerleau

This book describes a connectionist system called ALVINN (Autonomous Land Vehicle In a Neural Network) that overcomes difficulties and can learn to control an autonomous van in under 5 minutes by watching a person drive.

IEEE Transactions on Systems Man and CyberneticsNoise injection into inputs in back-propagation learning

257 Citations1992Kiyotoshi Matsuoka

The author mathematically explains why and how the noise injection to inputs has such an effect and experimentally shows experimentally that the generalization capability could remarkably be enhanced by training the network with noise injected inputs.

IEEE Transactions on Information TheoryThe relative value of labeled and unlabeled samples in pattern recognition with an unknown mixing parameter

206 Citations1996V. Castelli, Thomas M. Cover

It is shown that labeled samples are necessary to construct a classification rule and that they are exponentially more valuable than unlabeled samples.

Learning Classification with Unlabeled Data

204 Citations1993Virginia R. de

This paper shows that minimizing the disagreement between the outputs of networks processing patterns from these different modalities is a sensible approximation to minimizing the number of misclassifications in each modality, and leads to similar results.

Neural Information Processing SystemsInformation, Prediction, and Query by Committee

85 Citations1992Yoav Freund, H. Sebastian Seung +2 more

It is shown that if the two-member committee algorithm achieves information gain with positive lower bound, then the prediction error decreases exponentially with the number of queries, and this exponential decrease holds for query learning of thresholded smooth functions.

Physica D Nonlinear PhenomenaA stochastic version of the delta rule

62 Citations1990Stephen José Hanson

The delta rule is modified to include synaptic noise in the transmission of information and modification of the connection strength, and this stochastic version of the delta rule seems to promote escape from poor locally stable minima, and can improve convergence speed and likehood.

Nets with Unreliable Hidden Nodes Learn Error-Correcting Codes

21 Citations1992Stephen Judd, Paul Munro

In a multi-layered neural network, any one of the hidden layers can be viewed as computing a distributed representation of the input, but computing with such a representation requires completely dependable nodes that are noisy and unreliable.