Semi-Supervised Learning Literature Survey
Minds at UW (University of Wisconsin)Published 1 January 2005Open access
Xiaojin Zhu
Citations3,868
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The study clearly indicates that the common practice of stripwise precommercial thinning is unjustified, and the justification of heavy 'chessboard' thinning (with pruning) depends on whether the potential reduction in rotation length and the improvement in wood quality outweigh the discounted costs of pre-commercial thinning and selection and pruning of crop trees.
Abstract
We review some of the literature on semi-supervised learning in this paper. Traditional classifiers need labeled data (feature / label pairs) to train. Labeled instances however are often difficult, expensive, or time consuming to obtain, as they require the efforts of experienced human annotators. Meanwhile unlabeled data may be relatively easy to collect, but there has been few ways to use them. Semi-supervised learning addresses this problem by using large amount of unlabeled data, together with the labeled data, to build better classifiers. Because semi-supervised learning requires less human effort and gives higher \naccuracy, it is of great interest both in theory and in practice.
Keywords
Computer Science
Journal of the Royal Statistical Society Series B (Statistical Methodology)Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
49,657 Citations1977A. P. Dempster, N. M. Laird +1 more
Journal of Machine Learning ResearchLatent dirichlet allocation
27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Proceedings of the IEEEA tutorial on hidden Markov models and selected applications in speech recognition
22,785 Citations1989L. R. Rabiner
IEEE Transactions on Pattern Analysis and Machine IntelligenceNormalized cuts and image segmentation
15,696 Citations2000Jianbo Shi, Jitendra Malik
ScienceNonlinear Dimensionality Reduction by Locally Linear Embedding
15,035 Citations2000Sam T. Roweis, Lawrence K. Saul
Locally linear embedding (LLE) is introduced, an unsupervised learning algorithm that computes low-dimensional, neighborhood-preserving embeddings of high-dimensional inputs that learns the global structure of nonlinear manifolds.
ScienceA Global Geometric Framework for Nonlinear Dimensionality Reduction
13,740 Citations2000Joshua B. Tenenbaum, Vin de Silva +1 more
An approach to solving dimensionality reduction problems that uses easily measured local metric information to learn the underlying global geometry of a data set and efficiently computes a globally optimal solution, and is guaranteed to converge asymptotically to the true structure.
On Spectral Clustering: Analysis and an algorithm
7,751 Citations2001Andrew Y. Ng, Michael I. Jordan +1 more
A simple spectral clustering algorithm that can be implemented using a few lines of Matlab is presented, and tools from matrix perturbation theory are used to analyze the algorithm, and give conditions under which it can be expected to do well.
Neural ComputationLaplacian Eigenmaps for Dimensionality Reduction and Data Representation
7,709 Citations2003Mikhail Belkin, Partha Niyogi
This work proposes a geometrically motivated algorithm for representing the high-dimensional data that provides a computationally efficient approach to nonlinear dimensionality reduction that has locality-preserving properties and a natural connection to clustering.
Combining labeled and unlabeled data with co-training
5,604 Citations1998Avrim Blum, Tom M. Mitchell
The MIT Press eBooksSemi-Supervised Learning
4,308 Citations2006Olivier Chapelle, Schölkopf, B. +1 more
This first comprehensive overview of semi-supervised learning presents state-of-the-art algorithms, a taxonomy of the field, selected applications, benchmark experiments, and perspectives on ongoing and future research.
MPG.PuRe (Max Planck Society)Learning with Local and Global Consistency
3,750 Citations2003Dengyong Zhou, Olivier Bousquet +3 more
A principled approach to semi-supervised learning is to design a classifying function which is sufficiently smooth with respect to the intrinsic structure collectively revealed by known labeled and unlabeled points.
Semi-supervised learning using Gaussian fields and harmonic functions
3,377 Citations2003Xiaojin Zhu, Zoubin Ghahramani +1 more
A sentimental education
3,343 Citations2004Bo Pang, Lillian Lee
A novel machine-learning method is proposed that applies text-categorization techniques to just the subjective portions of the document, which greatly facilitates incorporation of cross-sentence contextual constraints.
A comparison of event models for naive bayes text classification
3,224 Citations1998Andrew McCallum, Kamal Nigam
It is found that the multi-variate Bernoulli performs well with small vocabulary sizes, but that the multinomial performs usually performs even better at larger vocabulary sizes--providing on average a 27% reduction in error over the multi -variateBernoulli model at any vocabulary size.
Machine LearningText Classification from Labeled and Unlabeled Documents using EM
2,749 Citations2000Kamal Nigam, Andrew Kachites McCallum +2 more
This paper shows that the accuracy of learned text classifiers can be improved by augmenting a small number of labeled training documents with a large pool of unlabeled documents, and presents two extensions to the algorithm that improve classification accuracy under these conditions.
Transductive Inference for Text Classification using Support Vector Machines
2,717 Citations1999Thorsten Joachims
An analysis of why TSVMs are well suited for text classi(cid:12)cation is presented, and an algorithm for training TSVMs e(cid:14)-ciently, handling 10,000 examples and more is proposed.
Unsupervised word sense disambiguation rivaling supervised methods
2,408 Citations1995David Yarowsky
An unsupervised learning algorithm for sense disambiguation that, when trained on unannotated English text, rivals the performance of supervised techniques that require time-consuming hand annotations.
arXiv (Cornell University)Probabilistic Latent Semantic Analysis
2,092 Citations2013Thomas Hofmann
This work proposes a widely applicable generalization of maximum likelihood model fitting by tempered EM, based on a mixture decomposition derived from a latent class model which results in a more principled approach which has a solid foundation in statistics.
Proceedings of the National Academy of SciencesHessian eigenmaps: Locally linear embedding techniques for high-dimensional data
1,696 Citations2003David L. Donoho, Carrie Grimes
The Hessian-based locally linear embedding method for recovering the underlying parametrization of scattered data (mi) lying on a manifold M embedded in high-dimensional Euclidean space is described, where the isometric coordinates can be recovered up to a linear isometry.
ACM Transactions on GraphicsColorization using optimization
1,439 Citations2004Anat Levin, Dani Lischinski +1 more
Journal of Machine Learning ResearchA Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data
1,372 Citations2005Rie Kubota Ando, Tong Zhang
This paper presents a general framework in which the structural learning problem can be formulated and analyzed theoretically, and relate it to learning with unlabeled data, and algorithms for structural learning will be proposed, and computational issues will be investigated.
IEEE Transactions on Pattern Analysis and Machine IntelligenceSpectral grouping using the nystrom method
1,326 Citations2004Charless C. Fowlkes, Serge Belongie +2 more
The contribution of this paper is a method that substantially reduces the computational requirements of grouping algorithms based on spectral partitioning making it feasible to apply them to very large grouping problems.
Exploiting Generative Models in Discriminative Classifiers
1,285 Citations1998Tommi Jaakkola, David Haussler
A natural way of achieving this combination by deriving kernel functions for use in discriminative methods such as support vector machines from generative probability models is developed.
Max-Margin Markov Networks
1,251 Citations2003Ben Taskar, Carlos Guestrin +1 more
Maximum margin Markov (M3) networks incorporate both kernels, which efficiently deal with high-dimensional features, and the ability to capture correlations in structured data, and a new theoretical bound for generalization in structured domains is provided.
The MIT Press eBooksLearning with Hypergraphs: Clustering, Classification, and Embedding
1,240 Citations2007Dengyong Zhou, Jiayuan Huang +1 more
This paper generalizes the powerful methodology of spectral clustering which originally operates on undirected graphs to hypergraphs, and further develop algorithms for hypergraph embedding and transductive classification on the basis of the spectral hypergraph clustering approach.
IEEE Transactions on Knowledge and Data EngineeringTri-training: exploiting unlabeled data using three classifiers
1,185 Citations2005Zhi‐Hua Zhou, Ming Li
Experiments on UCI data sets and application to the Web page classification task indicate that tri-training can effectively exploit unlabeled data to enhance the learning performance.
Analyzing the effectiveness and applicability of co-training
1,028 Citations2000Kamal Nigam, Rayid Ghani
It is demonstrated that when learning from labeled and unlabeled data, algorithms explicitly leveraging a natural independent split of the features outperform algorithms that do not and may out-perform algorithms not using a split.
Semi-supervised Learning by Entropy Minimization.
1,021 Citations2005Yves Grandvalet, Yoshua Bengio
This framework, which motivates minimum entropy regularization, enables to incorporate unlabeled data in the standard supervised learning, and includes other approaches to the semi-supervised problem as particular or limiting cases.
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)Learning from Labeled and Unlabeled Data using Graph Mincuts
947 Citations2018Avrim Blum, Shuchi Chawla
An algorithm based on finding minimum cuts in graphs, that uses pairwise relationships among the examples in order to learn from both labeled and unlabeled data is considered.
Lecture notes in computer scienceKernels and Regularization on Graphs
839 Citations2003Alexander J. Smola, Risi Kondor
It is shown that the class of positive, monotonically decreasing functions on the unit interval leads to kernels and corresponding regularization operators and can be found as a special case of the reasoning.
Unsupervised Models for Named Entity Classification
817 Citations1999Michael Collins, Yoram Singer
It is shown that the use of unlabeled data can reduce the requirements for supervision to just 7 simple "seed" rules, gaining leverage from natural redundancy in the data.
Semi-Supervised Support Vector Machines
793 Citations1998Kristin P. Bennett, Ayhan Demiriz
A general S3VM model is proposed that minimizes both the misclassification error and the function capacity based on all the available data that can be converted to a mixed-integer program and then solved exactly using integer programming.
Semi-Supervised Self-Training of Object Detection Models
791 Citations2005Charles Rosenberg, Martial Hebert +1 more
The key contributions of this empirical study are to demonstrate that a model trained in this manner can achieve results comparable to a modeltrained in the traditional manner using a much larger set of fully labeled data, and that a training data selection metric that is defined independently of the detector greatly outperforms a selection metric based on the detection confidence generated by the detector.
Diffusion Kernels on Graphs and Other Discrete Input Spaces
732 Citations2002Risi Kondor, John Lafferty
This paper proposes a general method of constructing natural families of kernels over discrete structures, based on the matrix exponentiation idea, and focuses on generating kernels on graphs, for which a special class of exponential kernels called diffusion kernels are proposed.
Segmentation using eigenvectors: a unifying view
722 Citations1999Yaakov Weiss
A unified treatment of eigenvectors of block matrices based on eigendecompositions in the context of segmentation is given, and close connections between them are shown while highlighting their distinguishing features.
Semi-Supervised Classification by Low Density Separation
710 Citations2005Olivier Chapelle, Alexander Zien +1 more
Three semi-supervised algorithms are proposed: deriving graph-based distances that emphazise low density regions between clusters, followed by training a standard SVM, and optimizing the Transductive SVM objective function by gradient descent.
MPG.PuRe (Max Planck Society)Ranking on Data Manifolds
673 Citations2003Dengyong Zhou, Jason Weston +3 more
A simple universal ranking algorithm for data lying in the Euclidean space, such as text or image data, to rank the data with respect to the intrinsic manifold structure collectively revealed by a great amount of data.
Proceedings of the IEEEFace Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About
672 Citations2006Pawan Sinha, Benjamin Balas +2 more
Findings from experimental studies of face recognition by humans provide insights into the nature of cues that the human visual system relies upon for achieving its impressive performance and serve as the building blocks for efforts to artificially emulate these abilities.
Semi-supervised learning with graphs
649 Citations2005Xiaojin Zhu, John Lafferty +1 more
A series of novel semi-supervised learning approaches arising from a graph representation, where labeled and unlabeled instances are represented as vertices, and edges encode the similarity between instances are presented.
Transductive learning via spectral graph partitioning
649 Citations2003Thorsten Joachims
This work proposes an algorithm that robustly achieves good generalization performance and that can be trained efficiently, and shows a connection to transductive Support Vector Machines, and that an effective Co-Training algorithm arises as a special case.
IEEE Transactions on Geoscience and Remote SensingThe effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon
572 Citations1994B.M. Shahshahani, D. A. Landgrebe
By using additional unlabeled samples that are available at no extra cost, the performance may be improved, and therefore the Hughes phenomenon can be mitigated and therefore more representative estimates can be obtained.
Partially labeled classification with Markov random walks
563 Citations2001Martin Szummer, Tommi Jaakkola
This work combines a limited number of labeled examples with a Markov random walk representation over the unlabeled examples and develops and compares several estimation criteria/algorithms suited to this representation.
Lecture notes in computer scienceRegularization and Semi-supervised Learning on Large Graphs
543 Citations2004Mikhail Belkin, Irina Matveeva +1 more
This work considers the problem of labeling a partially labeled graph, which may arise in a number of situations from survey sampling to information retrieval to pattern recognition in manifold settings.
The Annals of StatisticsConsistency of spectral clustering
535 Citations2008Ulrike von Luxburg, Mikhail A. Belkin +1 more
It is proved that one of the two major classes of spectral clustering (normalized clustering) converges under very general conditions, while the other is only consistent under strong additional assumptions, which are not always satisfied in real data.
Learning a kernel matrix for nonlinear dimensionality reduction
521 Citations2004Kilian Q. Weinberger, Fei Sha +1 more
This work investigates how to learn a kernel matrix for high dimensional data that lies on or near a low dimensional manifold and shows how to discover a mapping that "unfolds" the underlying manifold from which the data was sampled.
Partially Supervised Classification of Text Documents
516 Citations2002Bing Liu, Wee Sun Lee +2 more
This paper shows that the problem of identifying documents from a set of documents of a particular topic or class P and a large set M of mixed documents, and that under appropriate conditions, solutions to the constrained optimization problem will give good solution to the partially supervised classification problem.
Learning subjective nouns using extraction pattern bootstrapping
499 Citations2003Ellen Riloff, Janyce Wiebe +1 more
The goal of the research is to develop a system that can distinguish subjective sentences from objective sentences, and a Naive Bayes classifier is trained using the subjective nouns, discourse features, and subjectivity clues identified in prior research.
Integrating Topics and Syntax
492 Citations2004Thomas L. Griffiths, Mark Steyvers +2 more
This work presents a generative model that uses both kinds of dependencies, and can be used to simultaneously find syntactic classes and semantic topics despite having no representation of syntax or semantics beyond statistical dependency.
Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions
471 Citations2003Xiaojin Zhu, John Lafferty +1 more
The active learning scheme requires a much smaller number of queries to achieve high accuracy compared with random query selection, and the semi-supervised learning scheme requires a much smaller number of queries to achieve high accuracy compared with random query selection.
Encyclopedia of Machine Learning and Data MiningLearning from Labeled and Unlabeled Data
469 Citations2017Claude Sammut, Geoffrey I. Webb
Cluster Kernels for Semi-Supervised Learning
461 Citations2002Olivier Chapelle, Jason Weston +1 more
A framework to incorporate unlabeled data in kernel classifier, based on the idea that two points in the same cluster are more likely to have the same label is proposed by modifying the eigenspectrum of the kernel matrix.
Enhancing Supervised Learning with Unlabeled Data
459 Citations2000Sally A. Goldman, Yan Zhou
A new semi-supervised learning method called co-learning that is designed to use unlabeled data to enhance standard supervised learning algorithms to leverage off the fact that they have different representations of the hypotheses and are likely to detect different patterns in labeled data.
Beyond the point cloud
443 Citations2005Vikas Sindhwani, Partha Niyogi +1 more
This paper constructs a family of data-dependent norms on Reproducing Kernel Hilbert Spaces (RKHS) that allow the structure of the RKHS to reflect the underlying geometry of the data.
Bulletin of the London Mathematical SocietySPECTRAL GRAPH THEORY (CBMS Regional Conference Series in Mathematics 92)
438 Citations1998Norman Biggs
Learning from labeled and unlabeled data on a directed graph
403 Citations2005Dengyong Zhou, Jiayuan Huang +1 more
A general framework for learning from labeled and unlabeled data on a directed graph in which the structure of the graph including the directionality of the edges is considered, which generalizes the spectral clustering approach for undirected graphs.
Scientific Repository (Petra Christian University)Gaussian Processes for Ordinal Regression
373 Citations2005Wei Chu, Zoubin Ghahramani
Trading convexity for scalability
361 Citations2006Ronan Collobert, Fabian H. Sinz +2 more
It is shown how concave-convex programming can be applied to produce faster SVMs where training errors are no longer support vectors, and much faster Transductive SVMs.
Applied Physics Letters10.1162/153244304322972667
353 Citations2000
Locally linear embedding (LLE) is described, an unsupervised learning algorithm that computes low dimensional, neighborhood preserving embeddings of high dimensional data that relates to previous and ongoing work in nonlinear dimensionality reduction.
Learning with positive and unlabeled examples using weighted logistic regression
334 Citations2003Wee Sun Lee, Bing Liu
A performance measure that can be estimated from positive and unlabeled examples for evaluating retrieval performance, which is proportional to the product of precision and recall, can be used with a validation set to select regularization parameters for logistic regression.
Brunel University Research Archive (BURA) (Brunel University London)Two view learning: SVM-2K, Theory and Practice
331 Citations2005Jason Farquhar, David R. Hardoon +3 more
On Manifold Regularization.
323 Citations2005Misha Belkin, Partha Niyogi +1 more
A semisupervised framework that incorporates labeled and unlabeled data in a generalpurpose learner and gives rise to a regularized form of spectral clustering with an out-of-sample extension is proposed.
Seeing stars when there aren't many stars
319 Citations2006Andrew B. Goldberg, Xiaojin Zhu
A graph-based semi-supervised learning algorithm is presented to address the sentiment analysis task of rating inference and achieves significantly better predictive accuracy over other methods that ignore the unlabeled examples during training.
Neural Information Processing SystemsA Mixture of Experts Classifier with Learning Based on Both Labelled and Unlabelled Data
289 Citations1996David J. Miller, Hasan S. Uyar
A classifier structure and learning algorithm that make effective use of unlabelled data to improve performance and is a "mixture of experts" structure that is equivalent to the radial basis function (RBF) classifier, but unlike RBFs, is amenable to likelihood-based training.
Active + Semi-supervised Learning = Robust Multi-View Learning
288 Citations2002Ion Muslea, Steven Minton +1 more
A new multi-view algorithm, Co-EMT, which combines semi-supervised and active learning is introduced, which outperforms the other algorithms both on the parameterized problems and on two additional real world domains.
The MIT Press eBooksPAC Generalization Bounds for Co-training
287 Citations2002Sanjoy Dasgupta, Michael L. Littman +1 more
A new PAC-style bound on generalization error is given which justifies both the use of confidences — partial rules and partial labeling of the unlabeled data — and theUse of an agreement-based objective function as suggested by Collins and Singer.
Semi-supervised learning using randomized mincuts
268 Citations2004Avrim Blum, John Lafferty +2 more
The experiments on several datasets show that when the structure of the graph supports small cuts, this can result in highly accurate classifiers with good accuracy/coverage tradeoffs, and can be given theoretical justification from both a Markov random field perspective and from sample complexity considerations.
Label propagation through linear neighborhoods
261 Citations2006Fei Wang, Changshui Zhang
A novel graph-based semi supervised learning approach is proposed based on a linear neighborhood model, which assumes that each data point can be linearly reconstructed from its neighborhood, and can propagate the labels from the labeled points to the whole data set using these linear neighborhoods with sufficient smoothness.
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)Co-Training and Expansion: Towards Bridging Theory and Practice
261 Citations2018Maria-Florina Balcan, Avrim Blum +1 more
A much weaker "expansion" assumption on the underlying data distribution is proposed, that is proved to be sufficient for iterative co-training to succeed given appropriately strong PAC-learning algorithms on each feature set, and that to some extent is necessary as well.
Machine LearningMaximum Entropy Discrimination
259 Citations2004Tony Jebara
A general framework for discriminative estimation based on the maximum entropy principle and its extensions is presented and preliminary experimental results are indicative of the potential in these techniques.
International Joint Conference on Artificial IntelligenceSemi-supervised regression with co-training
244 Citations2005Zhi‐Hua Zhou, Ming Li
Experiments show that COREG can effectively exploit unlabeled data to improve regression estimates and is proposed as a co-training style semi-supervised regression algorithm.
Journal of Intelligent Information SystemsLatent Semantic Kernels
227 Citations2002Nello Cristianini, John Shawe‐Taylor +1 more
This paper describes how the LSI approach can be implemented in a kernel-defined feature space and provides experimental results demonstrating that the approach can significantly improve performance, and that it does not impair it.
Proceedings of the National Academy of SciencesEffects of temporal association on recognition memory
214 Citations2001Guy Wallis, HH Bülthoff
Results suggest that the authors are continuously associating views of objects to support later recognition, and that they do so not only on the basis of the physical similarity, but also the correlated appearance in time of the objects.
Unsupervised learning of image manifolds by semide .nite programming
214 Citations2004Kilian Q. Weinberger, L.K. Saul
Inference with the Universum
208 Citations2006Jason Weston, Ronan Collobert +3 more
An algorithm is described to leverage the Universum by maximizing the number of observed contradictions, and it is shown experimentally that this approach delivers accuracy improvements over using labeled data alone.
IEEE Transactions on Information TheoryThe relative value of labeled and unlabeled samples in pattern recognition with an unknown mixing parameter
206 Citations1996V. Castelli, Thomas M. Cover
It is shown that labeled samples are necessary to construct a classification rule and that they are exponentially more valuable than unlabeled samples.
Optimization methods & softwareSemi-superyised support vector machines for unlabeled data classification
204 Citations2001Glenn Fung, O. L. Mangasarian
Learning Classification with Unlabeled Data
204 Citations1993Virginia R. de
This paper shows that minimizing the disagreement between the outputs of networks processing patterns from these different modalities is a sensible approximation to minimizing the number of misclassifications in each modality, and leads to similar results.
Pattern Recognition LettersOn the exponential value of labeled samples
201 Citations1995Vittorio Castelli, Thomas M. Cover
The first labeled sample reduces the risk from 1 2 to 2R ∗ (1−R∗ ) and subsequent labeled samples in the training set reduce the probability of error exponentially fast to the Bayes risk.
Semi-supervised learning of mixture models
192 Citations2003Fábio Gagliardi Cozman, Ira L. Cohen +1 more
This paper analyzes the performance of semi-supervised learning of mixture models and shows that unlabeled data can lead to an increase in classification error even in situations where additional labeled data would decrease classification error.
A continuation method for semi-supervised SVMs
185 Citations2006Olivier Chapelle, Mingmin Chi +1 more
This paper proposes to use a global optimization technique known as continuation to alleviate the problem of non-convex optimization of S3VMs, which often results in suboptimal performances.
Lecture notes in computer scienceMulti-label Image Segmentation for Medical Applications Based on Graph-Theoretic Electrical Potentials
184 Citations2004Leo Grady, Gareth Funka-Lea
A novel method is proposed for performing multi-label, semi-automated image segmentation using combinatorial analogues of standard operators and principles from continuous potential theory, allowing it to be applied in arbitrary dimension.
Semi-supervised Learning on Directed Graphs
175 Citations2004Dengyong Zhou, Thomas Hofmann +1 more
A regularization framework for functions defined over nodes of a directed graph that forces the classification function to change slowly on densely linked subgraphs is proposed.
Érudit documents and data repository (Érudit Consortium, University of Montreal)Efficient Non-Parametric Function Induction in Semi-Supervised Learning
173 Citations2004Yoshua Bengio, Olivier Delalleau +1 more
Experiments show that the proposed non-parametric algorithms which provide an estimated continuous label for the given unlabeled examples are extended to function induction algorithms that correspond to the minimization of a regularization criterion applied to an out-of-sample example, and happens to have the form of a Parzen windows regressor.
Harmonic mixtures
169 Citations2005Xiaojin Zhu, John Lafferty
Experimental results show that this approach preserves the accuracy of purely graph-based transductive methods when the data has "manifold structure," and at the same time achieves inductive learning with significantly reduced computational cost.
KiltHub RepositoryNonparametric Transforms of Graph Kernels for Semi-Supervised Learning
164 Citations2018Xiaojin Zhu, Jaz Kandola +2 more
An algorithm based on convex optimization for constructing kernels for semi-supervised learning that incorporates order constraints during optimization results in flexible kernels and avoids the need to choose among different parametric forms.
ACM Transactions on Information SystemsEnhancing relevance feedback in image retrieval using unlabeled data
154 Citations2006Zhi‐Hua Zhou, Kejia Chen +1 more
Experiments show that using semisupervised learning and active learning simultaneously in CBIR is beneficial, and the proposed method achieves better performance than some existing methods.
Nonlinear Dimensionality Reduction by Semidefinite Programming and Kernel Matrix Factorization.
154 Citations2005Kilian Q. Weinberger, Benjamin Packer +1 more
It is shown that the full kernel matrix can be very well approximated by a product of smaller matrices, which leads to order-of-magnitude reductions in computation time and makes it possible to study much larger problems in manifold learning.
Unsupervised and semi-supervised multi-class support vector machines
151 Citations2005Linli Xu, Dale Schuurmans
A principled approach to unsupervised SVM training is presented by formulating convex relaxations of the natural training criterion: find a labeling that would yield an optimal SVM classifier on the resulting training data.
The Role of Unlabeled Data in Supervised Learning
150 Citations2004Tom M. Mitchell
It is argued that models of human and animal learning should consider more strongly the potential role of unlabeled data, and that many natural learning problems fit the problem class identified in this paper.
Neural Information Processing SystemsSemi-supervised Learning via Gaussian Processes
149 Citations2004Neil D. Lawrence, Michael I. Jordan
A probabilistic approach to learning a Gaussian Process classifier in the presence of unlabeled data using a "null category noise model" (NCNM) inspired by ordered categorical noise models.
The MIT Press eBooksManifold Denoising
148 Citations2007Matthias Hein, Markus Maier
The presented denoising algorithm is based on a graph-based diffusion process of the point sample and is analyzed using recent results about the convergence of graph Laplacians to improve the results of a semi-supervised learning algorithm.
Automatic Alignment of Local Representations
148 Citations2002Yee Whye Teh, Sam T. Roweis
An automatic alignment procedure which maps the disparate internal representations learned by several local dimensionality reduction experts into a single, coherent global coordinate system for the original data space is presented.
Munich Personal RePEc Archive (Ludwig Maximilian University of Munich)Using unlabeled data to improve text classification
129 Citations2001Kanal Paul Nigam, Tom M. Mitchell
This dissertation demonstrates that supervised learning algorithms that use a small number of labeled examples and many inexpensive unlabeled examples can create high-accuracy text classifiers.
Does Baum-Welch re-estimation help taggers?
129 Citations1994David Elworthy
Two experiments designed to determine how much manual training information is needed for speech tagging by Hidden Markov Model suggest that initial biasing of either lexical or transition probabilities is essential to achieve a good accuracy and reveal three distinct patterns of Baum-Welch reestimation.
Deterministic annealing for semi-supervised kernel machines
129 Citations2006Vikas Sindhwani, S. Sathiya Keerthi +1 more
This paper presents a global optimization framework for semi-supervised Kernel machines where an easier problem is parametrically deformed to the original hard problem and minimizers are smoothly tracked.
The MIT Press eBooksBranch and Bound for Semi-Supervised Support Vector Machines
129 Citations2007Olivier Chapelle, Vikas Sindhwani +1 more
Empirical evidence suggests that the globally optimal solution of S3VMs modulo local minima problems in current implementations can return excellent generalization performance in situations where other implementations fail completely.
Kernel conditional random fields
124 Citations2004John Lafferty, Xiaojin Zhu +1 more
Kernel conditional random fields are introduced as a framework for discriminative modeling of graph-structured data and a procedure for greedily selecting cliques in the dual representation is proposed, which allows sparse representations.
Semi-supervised nonlinear dimensionality reduction
124 Citations2006Xin Yang, Haoying Fu +2 more
The sensitivity analysis of the algorithms shows that prior information will improve stability of the solution, and some insight is given on what kind of prior information best improves the solution.
Neural Information Processing SystemsProximity Graphs for Clustering and Manifold Learning
120 Citations2004Richard S. Zemel, Miguel Á. Carreira-Perpiñán
It is shown that a graph ensemble usually produces a better representation of the data manifold than standard methods; and that it provides robustness to a subsequent clustering or dimensionality reduction algorithm based on the graph.
…