Multi-view Discriminative Sequential Learning
Lecture notes in computer sciencePublished 1 January 2005Open access
Ulf Brefeld, Christoph Büscher, Tobias Scheffer
Citations47
SJR quartileQ2
SJR score0.35
SNIP0.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The multi-view approach is based on the principle of maximizing the consensus among multiple independent hypotheses and develops this principle into a semi-supervised hidden Markov perceptron, and the resulting procedures utilize unlabeled data effectively and discriminate more accurately than their purely supervised counterparts.
Abstract
Discriminative learning techniques for sequential data have proven to be more effective than generative models for named entity recognition, information extraction, and other tasks of discrimination. However, semi-supervised learning mechanisms that utilize inexpensive unlabeled sequences in addition to few labeled sequences – such as the Baum-Welch algorithm – are available only for generative models. The multi-view approach is based on the principle of maximizing the consensus among multiple independent hypotheses; we develop this principle into a semi-supervised hidden Markov perceptron, and a semi-supervised hidden Markov support vector learning algorithm. Experiments reveal that the resulting procedures utilize unlabeled data effectively and discriminate more accurately than their purely supervised counterparts.
Keywords
Computer Science
NeurocomputingAdvances in neural information processing systems 7
22,300 Citations1997Krzysztof J. Cios, Mark E. Shields
ScholarlyCommons (University of Pennsylvania)Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
12,978 Citations2001John Lafferty, Andrew McCallum +1 more
This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
Combining labeled and unlabeled data with co-training
5,604 Citations1998Avrim Blum, Tom M. Mitchell
Unsupervised word sense disambiguation rivaling supervised methods
2,408 Citations1995David Yarowsky
An unsupervised learning algorithm for sense disambiguation that, when trained on unannotated English text, rivals the performance of supervised techniques that require time-consuming hand annotations.
Maximum Entropy Markov Models for Information Extraction and Segmentation
1,333 Citations2000Andrew McCallum, Dayne Freitag +1 more
A new Markovian sequence model is presented that allows observations to be represented as arbitrary overlapping features (such as word, capitalization, formatting, part-of-speech), and defines the conditional probability of state sequences given observation sequences.
Support vector machine learning for interdependent and structured output spaces
1,260 Citations2004Ioannis Tsochantaridis, Thomas Hofmann +2 more
This paper proposes to generalize multiclass Support Vector Machine learning in a formulation that involves features extracted jointly from inputs and outputs, and demonstrates the versatility and effectiveness of the method on problems ranging from supervised grammar learning and named-entity recognition, to taxonomic text classification and sequence alignment.
Max-Margin Markov Networks
1,251 Citations2003Ben Taskar, Carlos Guestrin +1 more
Maximum margin Markov (M3) networks incorporate both kernels, which efficiently deal with high-dimensional features, and the ability to capture correlations in structured data, and a new theoretical bound for generalization in structured domains is provided.
Analyzing the effectiveness and applicability of co-training
1,028 Citations2000Kamal Nigam, Rayid Ghani
It is demonstrated that when learning from labeled and unlabeled data, algorithms explicitly leveraging a natural independent split of the features outperform algorithms that do not and may out-perform algorithms not using a split.
Unsupervised Models for Named Entity Classification
817 Citations1999Michael Collins, Yoram Singer
It is shown that the use of unlabeled data can reduce the requirements for supervision to just 7 simple "seed" rules, gaining leverage from natural redundancy in the data.
The MIT Press eBooksConvolution Kernels for Natural Language
812 Citations2002Michael J. Collins, Nigel Duffy
It is shown how a kernel over trees can be applied to parsing using the voted perceptron algorithm, and experimental results on the ATIS corpus of parse trees are given.
New ranking algorithms for parsing and tagging
505 Citations2001Michael Collins, Nigel Duffy
New learning algorithms for natural language processing based on the perceptron algorithm are introduced, showing how the algorithms can be efficiently applied to exponential sized representations of parse trees, such as the "all subtrees" (DOP) representation described by (Bod 1998).
Hidden Markov support vector machines
469 Citations2003Yasemin Altün, Ioannis Tsochantaridis +1 more
This paper presents a novel discriminative learning technique for label sequences based on a combination of the two most successful learning algorithms, Support Vector Machines and Hidden Markov Models which it is called HM-SVMs and handles dependencies between neighboring labels using Viterbi decoding.
Active + Semi-supervised Learning = Robust Multi-View Learning
288 Citations2002Ion Muslea, Steven Minton +1 more
A new multi-view algorithm, Co-EMT, which combines semi-supervised and active learning is introduced, which outperforms the other algorithms both on the parameterized problems and on two additional real world domains.
The MIT Press eBooksPAC Generalization Bounds for Co-training
287 Citations2002Sanjoy Dasgupta, Michael L. Littman +1 more
A new PAC-style bound on generalization error is given which justifies both the use of confidences — partial rules and partial labeling of the unlabeled data — and theUse of an agreement-based objective function as suggested by Collins and Singer.
Ranking algorithms for named-entity extraction
241 Citations2001Michael Collins
Algorithms which rerank the top N hypotheses from a maximum-entropy tagger, the application being the recovery of named-entity boundaries in a corpus of web data, using the voted perceptron algorithm.
BMC BioinformaticsIdentifying gene and protein mentions in text using conditional random fields
223 Citations2005Ryan McDonald, Fernando Pereira
A diverse feature set containing standard orthographic features combined with expert features in the form of gene and biological term lexicons is employed to achieve a precision of 86.4% and recall of 78.7% for tagging gene and protein mentions from text using the probabilistic sequence tagging framework of conditional random fields.
Learning Classification with Unlabeled Data
204 Citations1993Virginia R. de
This paper shows that minimizing the disagreement between the outputs of networks processing patterns from these different modalities is a sensible approximation to minimizing the number of misclassifications in each modality, and leads to similar results.
Co-EM support vector learning
200 Citations2004Ulf Brefeld, Tobias Scheffer
This work casts linear classifiers into a probabilistic framework and develops a co-EM version of the Support Vector Machine, which conducts experiments on text classification problems and compares the family of semi-supervised support vector algorithms under different conditions, including violations of the assumptions underlying multi-view learning.
Bootstrapping
148 Citations2001Steven Abney
This chapter develops a new conception of the bootstrapping objection, according to which the problem with the idea that intentions provide reasons is that the problem with the idea that intentions provide reasons is that the authors find ourselves with a normative theory that issues incoherent advice.
Kernel conditional random fields
124 Citations2004John Lafferty, Xiaojin Zhu +1 more
Kernel conditional random fields are introduced as a framework for discriminative modeling of graph-structured data and a procedure for greedily selecting cliques in the dual representation is proposed, which allows sparse representations.
Combining Labeled and Unlabeled Data for MultiClass Text Categorization
104 Citations2002Rayid Ghani
This paper develops a framework to incorporate unlabeled data in the Error-Correcting Output Coding (ECOC) setup by first decomposing multiclass problems into multiple binary problems and then using Co-Training to learn the individual binary classi cation problems.
Gaussian process classification for segmenting and annotating sequences
53 Citations2004Yasemin Altün, Thomas Hofmann +1 more
Gaussian Process classification is generalized to predict multiple labels by taking dependencies between neighboring labels into account, motivated by the desire to retain rigorous probabilistic semantics, while overcoming limitations of parametric methods like Conditional Random Fields.
Uncertainty in Artificial IntelligenceCase-factor diagrams for structured probabilistic modeling
53 Citations2004David McAllester, Michael Collins +1 more
Discriminative Learning for Label Sequences via Boosting
51 Citations2002Yasemin Altün, Thomas Hofmann +1 more
The proposed sequence boosting algorithm offers an interesting alternative to methods based on HMMs and the more recently proposed Conditional Random Fields and is attractive both, conceptually and computationally.
BMC BioinformaticsSystematic feature evaluation for gene name recognition
45 Citations2005Jörg Hakenberg, Steffen Bickel +6 more
A systematic approach to evaluate the performance of different feature sets based on recursive feature elimination, RFE, which helps to identify descriptive features, to learn about the structure of the problem, and to design systems that are faster and easier to understand.
BMC BioinformaticsComputational Analysis of Insulin in the Ageing Fruit Fly
6 Citations2005John Cumbers, J. Douglas Armstrong
This work investigates the reduction of an insulin-like peptide that is caused by decreasing the amount of yeast fed to larval and adult fruit flies and Quantifying insulin production in this way and measuring its responsiveness to diet will enable to better model the functions of genes acting upstream and downstream of the insulin peptide and the insulin cell surface receptors.