Real-World Semi-Supervised Learning of POS-Taggers for Low-Resource Languages
Published 1 August 2013
Dan Garrette, Jason D. Mielens, Jason Baldridge
Citations48
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The results show that annotation of word types is the most important, provided a sufficiently capable semi-supervised learning infrastructure is in place to project type information onto a raw corpus and that finitestate morphological analyzers are effective sources of type information when few labeled examples are available.
Abstract
Developing natural language processing tools for low-resource languages often requires creating resources from scratch. While a variety of semi-supervised methods exist for training from incomplete data, there are open questions regarding what types of training data should be used and how much is necessary. We discuss a series of experiments designed to shed light on such questions in the context of part-of-speech tagging. We obtain timed annotations from linguists for the low-resource languages Kinyarwanda and Malagasy (as well as English) and evaluate how the amounts of various kinds of data affect performance of a trained POS-tagger. Our results show that annotation of word types is the most important, provided a sufficiently capable semi-supervised learning infrastructure is in place to project type information onto a raw corpus. We also show that finitestate morphological analyzers are effective sources of type information when few labeled examples are available. 1
Keywords
Computer Science
Building a Large Annotated Corpus of English: The Penn Treebank
7,528 Citations1993Mitchell P. Marcus
Tagging English text with a probabilistic model
478 Citations1994Bernard Mérialdo
Computer Speech & LanguageRobust part-of-speech tagging using a hidden Markov model
416 Citations1992Julian Kupiec
A system for part-of-speech tagging is described, based on a hidden Markov model which can be trained using a corpus of untagged text, which results in a model that correctly tags approximately 96% of the text.
Lecture notes in computer sciencePart-of-Speech Tagging from 97% to 100%: Is It Time for Some Linguistics?
408 Citations2011Christopher D. Manning
It is suggested and demonstrated that the largest opportunity for further progress comes from improving the taxonomic basis of the linguistic resources from which taggers are trained, that is, from improved descriptive linguistics.
Unsupervised Part-of-Speech Tagging with Bilingual Graph-Based Projections
247 Citations2011Dipanjan Das, Slav Petrov
A novel approach for inducing unsupervised part-of-speech taggers for languages that have no labeled training data, but have translated text in a resource-rich language, using graph-based label propagation for cross-lingual knowledge transfer.
Lecture notes in computer scienceNew Regularized Algorithms for Transductive Learning
210 Citations2009Partha Talukdar, Koby Crammer
This work proposes a new graph-based label propagation algorithm for transductive learning that can be extended to incorporate additional prior information, and demonstrates it with classifying data where the labels are not mutually exclusive.
Transactions of the Association for Computational LinguisticsToken and Type Constraints for Cross-Lingual Part-of-Speech Tagging
188 Citations2013Oscar Täckström, Dipanjan Das +3 more
It is shown that additional token constraints can be projected from a resource-rich source language to a resourceful target language via word-aligned bitext, and empirically demonstrating the applicability of coupled token and type constraints across a diverse set of languages.
Prototype-driven learning for sequence models
178 Citations2006Aria Haghighi, Dan Klein
This work investigates prototype-driven learning for primarily unsupervised sequence modeling, where prior knowledge is specified declaratively, by providing a few canonical examples of each target annotation label, then propagated across a corpus using distributional similarity features in a log-linear generative model.
The MIT Press eBooksDeterministic Part-of-Speech Tagging with Finite-State Transducers
136 Citations1997Emmanuel Roche, Yves Schabes
Detecting errors in part-of-speech annotation
120 Citations2003Markus Dickinson, Detmar Meurers
A new method is proposed for detecting errors in "gold-standard" part-of-speech annotation based on n-grams occurring in the corpus with multiple taggings based on closed-class analysis and finite-state tagging guide patterns.
Efficient Graph-Based Semi-Supervised Learning of Structured Tagging Models
118 Citations2010Amarnag Subramanya, Slav Petrov +1 more
A new scalable algorithm for semi-supervised training of conditional random fields (CRF) and its application to part-of-speech (POS) tagging that uses a similarity graph to encourage similar n-grams to have similar POS tags is described.
Neural Information Processing SystemsA Bayesian LDA-based model for semi-supervised part-of-speech tagging
112 Citations2007Kristina Toutanova, Mark Johnson
A novel Bayesian model for semi-supervised part-of-speech tagging that outperforms the best previously proposed model for this task on a standard dataset and introduces a model for determining the set of possible tags of a word which captures important dependencies in the ambiguity classes of words.
Learning a Part-of-Speech Tagger from Two Hours of Annotation
96 Citations2013Dan Garrette, Jason Baldridge
This paper attempts to create true low-resource scenarios by allowing a linguist just two hours to annotate data and evaluating on the languages Kinyarwanda and Malagasy through a method of automatically generalizing the annotations, reducing noise, and inducing word-tag frequency information.
Wiki-ly Supervised Part-of-Speech Tagging
84 Citations2012Shen Li, Joäo Graça +1 more
This paper shows that it is possible to build POS-taggers exceeding state-of-the-art bilingual methods by using simple hidden Markov models and a freely available and naturally growing resource, the Wiktionary.
Rule writing or annotation
82 Citations2000Grace Ngai, David Yarowsky
Results show that it is more efficient and more successful by several measures to train a system using active learning annotation rather than hand-crafted rule writing at a comparable level of human labor investment.
Minimized models for unsupervised part-of-speech tagging
79 Citations2009Sujith Ravi, Kevin Knight
A novel method is described that uses integer programming to explicitly search for the smallest model that explains the data, and then uses EM to set parameter values, and performs better than existing state-of-the-art systems in both settings.
EM Can Find Pretty Good HMM POS-Taggers (When Given a Good Start)
58 Citations2008Yoav Goldberg, Meni Adler +1 more
It is demonstrated that good results can be obtained using the robust EM-HMM learner when provided with good initial conditions, even with incomplete dictionaries, according to a family of algorithms to compute effective initial estimations p(t|w).
How well does active learning<i>actually</i>work?
53 Citations2009Jason Baldridge, Alexis Palmer
It is found that better example selection and label suggestions improveency, but effectiveness depends strongly on annotator expertise, and of the need for cost-sensitive learning meth-ods that adapt to annotators.
Lecture notes in computer scienceApplications of Finite-State Transducers in Natural Language Processing
48 Citations2001Lauri Karttunen
This paper is a review of some of the major applications of finite- state transducers in Natural Language Processing ranging from morphological analysis to finite-state parsing.
Bootstrapping a multilingual part-of-speech tagger in one person-day
47 Citations2002Silviu Cucerzan, David Yarowsky
The algorithm begins by inducing initial lexical POS distributions from English translations in a bilingual dictionary without POS tags, and handles irregular, regular and semi-regular morphology through a robust generative model using weighted Levenshtein alignments.
Type-Supervised Hidden Markov Models for Part-of-Speech Tagging with Incomplete Tag Dictionaries
23 Citations2012Dan Garrette, Jason Baldridge
Taking the min-greedy algorithm as a starting point, this work improves it with several intuitive heuristics and defines a simple HMM emission initialization that takes advantage of the tag dictionary and raw data to capture both the openness of a given tag and its estimated prevalence in the raw data.
Weakly supervised part-of-speech tagging for morphologically-rich, resource-scarce languages
14 Citations2009Kazi Saidul Hasan, Vincent Ng
This paper argues that existing unsupervised POS taggers unrealistically assume as input a perfect POS lexicon, and proposes a weakly supervised fully-Bayesian approach to POS tagging, which relaxes the unrealistic assumption by automatically acquiring the lexicon from a small amount of POS-tagged data.
Fast, Greedy Model Minimization for Unsupervised Tagging
10 Citations2010Sujith Ravi, Ashish Vaswani +2 more
This work proposes a novel two-stage greedy approximation scheme to replace the IP, and runs fast, while yielding highly accurate tagging results, and compares it against standard EM training, and shows that it consistently obtain better tagging accuracies on test data of varying sizes for English and Italian.
Linguistic Issues in Language TechnologyBootstrapping the Language Archive
8 Citations2011Steven Bird
A set of seven constraints (or "axioms for scalable work with small languages") are presented, and suggestions for further NLP research are related back to these axioms.