A New Analysis of Co-Training
Published 21 June 2010
Wei Wang, Zhi‐Hua Zhou
Citations174
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
In this analysis the co-training process is viewed as a combinative label propagation over two views; this provides a possibility to bring the graph-based and disagreement-based semi-supervised methods into a unified framework.
Abstract
In this paper, we present a new analysis on co-training, a representative paradigm of disagreement-based semi-supervised learning methods. In our analysis the co-training process is viewed as a combinative label propagation over two views; this provides a possibility to bring the graph-based and disagreementbased semi-supervised methods into a unified framework. With the analysis we get some insight that has not been disclosed by previous theoretical studies. In particular, we provide the sufficient and necessary condition for co-training to succeed. We also discuss the relationship to previous theoretical results and give some other interesting implications of our results, such as combination of weight matrices and view split. 1.
Keywords
Computer Science
Combining labeled and unlabeled data with co-training
5,604 Citations1998Avrim Blum, Tom M. Mitchell
The MIT Press eBooksSemi-Supervised Learning
4,308 Citations2006Olivier Chapelle, Schölkopf, B. +1 more
This first comprehensive overview of semi-supervised learning presents state-of-the-art algorithms, a taxonomy of the field, selected applications, benchmark experiments, and perspectives on ongoing and future research.
Minds at UW (University of Wisconsin)Semi-Supervised Learning Literature Survey
3,868 Citations2005Xiaojin Zhu
The study clearly indicates that the common practice of stripwise precommercial thinning is unjustified, and the justification of heavy 'chessboard' thinning (with pruning) depends on whether the potential reduction in rotation length and the improvement in wood quality outweigh the discounted costs of pre-commercial thinning and selection and pruning of crop trees.
IEEE Transactions on Knowledge and Data EngineeringTri-training: exploiting unlabeled data using three classifiers
1,185 Citations2005Zhi‐Hua Zhou, Ming Li
Experiments on UCI data sets and application to the Web page classification task indicate that tri-training can effectively exploit unlabeled data to enhance the learning performance.
Analyzing the effectiveness and applicability of co-training
1,028 Citations2000Kamal Nigam, Rayid Ghani
It is demonstrated that when learning from labeled and unlabeled data, algorithms explicitly leveraging a natural independent split of the features outperform algorithms that do not and may out-perform algorithms not using a split.
Semi-supervised learning with graphs
649 Citations2005Xiaojin Zhu, John Lafferty +1 more
A series of novel semi-supervised learning approaches arising from a graph representation, where labeled and unlabeled instances are represented as vertices, and edges encode the similarity between instances are presented.
Enhancing Supervised Learning with Unlabeled Data
459 Citations2000Sally A. Goldman, Yan Zhou
A new semi-supervised learning method called co-learning that is designed to use unlabeled data to enhance standard supervised learning algorithms to leverage off the fact that they have different representations of the hypotheses and are likely to detect different patterns in labeled data.
Spectral clustering and transductive learning with multiple views
452 Citations2007Dengyong Zhou, Christopher J. C. Burges
This work develops multiview spectral clustering via generalizing the normalized cut from a single view to multiple views and further builds multivView transductive inference on the basis of multiv View spectral clusters.
Knowledge and Information SystemsSemi-supervised learning by disagreement
416 Citations2009Zhi‐Hua Zhou, Ming Li
An introduction to research advances in disagreement-based semi-supervised learning is provided, where multiple learners are trained for the task and the disagreements among the learners are exploited during the semi-supervised learning process.
A Co-Regularization Approach to Semi-supervised Learning with Multiple Views
337 Citations2005Vikas Sindhwani, Partha Niyogi +1 more
This paper proposes a Co-Regularization framework where classifiers are learnt in each view through forms of multi-view regularization, and proposes algorithms within this framework that are based on optimizing measures of agreement and smoothness over labeled and unlabeled examples.
Graph construction and <i>b</i> -matching for semi-supervised learning
325 Citations2009Tony Jebara, Jun Wang +1 more
Experimental results on both artificial data and real benchmark datasets indicate that b-matching produces more robust graphs and therefore provides significantly better prediction accuracy without any significant change in computation time.
Active + Semi-supervised Learning = Robust Multi-View Learning
288 Citations2002Ion Muslea, Steven Minton +1 more
A new multi-view algorithm, Co-EMT, which combines semi-supervised and active learning is introduced, which outperforms the other algorithms both on the parameterized problems and on two additional real world domains.
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)Co-Training and Expansion: Towards Bridging Theory and Practice
261 Citations2018Maria-Florina Balcan, Avrim Blum +1 more
A much weaker "expansion" assumption on the underlying data distribution is proposed, that is proved to be sufficient for iterative co-training to succeed given appropriately strong PAC-learning algorithms on each feature set, and that to some extent is necessary as well.
Lecture notes in computer scienceAnalyzing Co-training Style Algorithms
219 Citations2007Wei Wang, Zhi‐Hua Zhou
It is shown that the co- training process can succeed even without two views, given that the two learners have large difference, which explains the success of some co-training style algorithms that do not require two views.
Bootstrapping statistical parsers from small datasets
153 Citations2003Mark Steedman, Miles Osborne +7 more
Experimental results show that unlabelled sentences can be used to improve the performance of statistical parsers and it is shown that boot-strapping continues to be useful, even though no manually produced parses from the target domain are used.
Influence of graph construction on graph-based clustering measures
151 Citations2008Markus Maier, Ulrike von Luxburg +1 more
This paper studies the convergence of graph clustering criteria such as the normalized cut (Ncut) as the sample size tends to infinity and finds that the limit expressions are different for different types of graph, for example the r-neighborhood graph or the k-nearest neighbor graph.
Combining Graph Laplacians for Semi--Supervised Learning
126 Citations2005Andreas A. Argyriou, Mark Herbster +1 more
This work proposes to use a method which optimally combines a number of differently constructed graphs to solve an extended regularization problem which requires a joint minimization over both the data and the set of graph kernels.
Linear prediction models with graph regularization for web-page categorization
112 Citations2006Tong Zhang, Alexandrin Popescul +1 more
A risk minimization formulation for learning from both text and graph structures which is motivated by the problem of collective inference for hypertext document categorization is presented, based on graph regularization formulated as a well-formed convex optimization problem.
Conference on Learning TheoryDoes Unlabeled Data Provably Help? Worst-case Analysis of the Sample Complexity of Semi-Supervised Learning.
108 Citations2008Shai Ben-David, Tyler Lu +1 more
It is proved that for basic hypothesis classes over the real line, if the distribution of unlabeled data is ‘smooth’, knowledge of that distribution cannot improve the labeled sample complexity by more than a constant factor.
Bayesian Co-Training
56 Citations2007Shipeng Yu, Balaji Krishnapuram +3 more
Edinburgh Research Explorer (University of Edinburgh)Corrected Co-training for Statistical Parsers
55 Citations2003Rebecca Hwa, Miles Osborne +2 more
This work proposes a novel approach, which it is shown that this method requires only a third as many manual annotation decisions as corrected co-training/co-testing to achieve the same improvement in performance.
Lecture notes in computer scienceMulti-view Discriminative Sequential Learning
47 Citations2005Ulf Brefeld, Christoph Büscher +1 more
The multi-view approach is based on the principle of maximizing the consensus among multiple independent hypotheses and develops this principle into a semi-supervised hidden Markov perceptron, and the resulting procedures utilize unlabeled data effectively and discriminate more accurately than their purely supervised counterparts.