Part-of-Speech Tagging and Partial Parsing
Text, speech and language technologyPublished 1 January 1997
Steven Abney
Citations186
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The initial impetus for the current popularity of statistical methods in computational linguistics was provided in large part by the papers on part-of-speech tagging by Church, DeRose, and Garside, which showed that it was indeed possible to carve part- of-speech disambiguation out of the apparently monolithic problem of natural language understanding, and solve it with impressive accuracy.
Abstract
The initial impetus for the current popularity of statistical methods in computational linguistics was provided in large part by the papers on part-of-speech tagging by Church (1988), DeRose (1988), and Garside (1987b). In contradiction to common wisdom, these taggers showed that it was indeed possible to carve part-of-speech disambiguation out of the apparently monolithic problem of natural language understanding, and solve it with impressive accuracy.
Keywords
Computer Science
BiometricsClassification and Regression Trees.
23,841 Citations1984Alexander Gordon, Leo Breiman +3 more
Proceedings of the IEEEA tutorial on hidden Markov models and selected applications in speech recognition
22,785 Citations1989L. R. Rabiner
Compilers: Principles, Techniques, and Tools
8,137 Citations1986Alfred V. Aho, Ravi Sethi +1 more
This book discusses the design of a Code Generator, the role of the Lexical Analyzer, and other topics related to code generation and optimization.
IEEE ASSP MagazineAn introduction to hidden Markov models
4,789 Citations1986L. R. Rabiner, Biing‐Hwang Juang
The purpose of this tutorial paper is to give an introduction to the theory of Markov models, and to illustrate how they have been applied to problems in speech recognition.
Class-based n -gram models of natural language
2,893 Citations1992Peter F. Brown, P.V. deSouza +3 more
This work addresses the problem of predicting a word from previous words in a sample of text and discusses n-gram models based on classes of words, finding that these models are able to extract classes that have the flavor of either syntactically based groupings or semanticallybased groupings, depending on the nature of the underlying statistics.
Communications of the ACMTransition network grammars for natural language analysis
1,330 Citations1970William A. Woods
Text, speech and language technologyText Chunking Using Transformation-Based Learning
1,257 Citations1999Lance Ramshaw, Mitchell P. Marcus
This work has shown that the transformation-based learning approach can be applied at a higher level of textual interpretation for locating chunks in the tagged text, including non-recursive “baseNP” chunks.
A stochastic parts program and noun phrase parser for unrestricted text
974 Citations1988Kenneth Church
A program that tags each word in an input sentence with the most likely part of speech has been written and performance is encouraging; a 400-word sample is presented and is judged to be 99.5% correct.
Language<b>A theory of syntactic recognition for natural language</b> . By Mitchell P. Marcus. Cambridge, MA: MIT Press, 1980. Pp. 335. $25.00.
919 Citations1982Michael B. Kac
It will be shown that this 'determinism' hypothesis, explored within the context of the grammar of English, leads to a simple mechanism, a grammar interpreter.
Studies in linguistics and philosophyParsing By Chunks
878 Citations1991Steven Abney
The typical chunk consists of a single content word surrounded by a constellation of function words, matching a fixed template, and the relationships between chunks are mediated more by lexical selection than by rigid templates.
Computational LinguisticsRegular models of phonological rule systems
816 Citations1994Ronald M. Kaplan, Martin Kay
This paper shows in detail how this framework applies to ordered sets of context-sensitive rewriting rules and also to grammars in Koskenniemi's two-level formalism.
Helda (University of Helsinki)Two-Level Morphology: A General Computational Model for Word-Form Recognition and Production
792 Citations1983Kimmo Koskenniemi
Machine LearningLearning Decision Lists
761 Citations1987Ronald L. Rivest
This paper introduces a new representation for Boolean functions, called decision lists, and shows that they are efficiently learnable from examples, and strictly increases the set of functions known to be polynomially learnable, in the sense of Valiant (1984).
A practical part-of-speech tagger
632 Citations1992Doug Cutting, Julian Kupiec +2 more
An implementation of a part-of-speech tagger based on a hidden Markov model that enables robust and accurate tagging with few resource requirements and accuracy exceeds 96%.
Natural Language EngineeringPartial parsing via finite-state cascades
480 Citations1996Steven Abney
Deterministic parsers specified by finite state cascades may be more accurate than exhaustive search stochastic context free parsers and extended at modest cost to construct parse trees with finite feature structures.
Tagging English text with a probabilistic model
478 Citations1994Bernard Mérialdo
Computational LinguisticsGrammatical category disambiguation by statistical optimization
387 Citations1988Steven J. DeRose
An algorithm for disambiguation that is similar to CLAWS but that operates in linear rather than in exponential time and space, and which minimizes the unsystematic augments is presented.
Decision lists for lexical ambiguity resolution
362 Citations1994David Yarowsky
This paper presents a statistical decision procedure for lexical ambiguity resolution that exploits both local syntactic patterns and more distant collocational evidence, generating an efficient, effective, and highly perspicuous recipe for resolving a given ambiguity.
From Phoneme to Morpheme
303 Citations1970Zellig S. Harris
arXiv (Cornell University)Some Advances in Transformation-Based Part of Speech Tagging
279 Citations1994Eric Brill
IEEE Transactions on Systems Man and CyberneticsGrammatical Inference: Introduction and Survey - Part I
272 Citations1975King‐Sun Fu, Taylor L. Booth
IEEE Transactions on Acoustics Speech and Signal ProcessingA tree-based statistical language model for natural language speech recognition
253 Citations1989L.R. Bahl, Peter F. Brown +2 more
Algorithms are presented for automatically constructing a binary decision tree designed to estimate the probability that a given word will be the next word uttered, which is compared to an equivalent trigram model and shown to be superior.
arXiv (Cornell University)Natural Language Parsing as Statistical Pattern Recognition
226 Citations1994David M. Magerman
This work proposes an automatic method for acquiring a statistical parser from a set of parsed sentences which takes advantage of some initial linguistic input, but avoids the pitfalls of the iterative and seemingly endless grammar development process.
Lecture notes in computer scienceInducing probabilistic grammars by Bayesian model merging
218 Citations1994Andreas Stolcke, Stephen M. Omohundro
A framework for inducing probabilistic grammars from corpora of positive samples is described, which formalizes a trade-off between a close fit to the data and a default preference for simpler models (‘Occam's Razor’).
An algorithm for finding noun phrase correspondences in bilingual corpora
210 Citations1993Julian Kupiec
The paper describes an algorithm that employs English and French text taggers to associate noun phrases in an aligned bilingual corpus and provides an alternative to other approaches for finding word correspondences, with the advantage that linguistic structure is incorporated.
Automatic acquisition of a large subcategorization dictionary from corpora
206 Citations1993Christopher D. Manning
It is shown that statistical filtering of the results of a finite state parser running on the output of a stochastic tagger produces high quality results, despite the error rates of the tagger and the parser.
Constraint grammar as a framework for parsing running text
206 Citations1990Fred Karlsson
This work presents a formalism to be used for parsing where the grammar statements are closer to real text sentences and more directly address some notorious parsing problems, especially ambiguity.
Surface grammatical analysis for the extraction of terminological noun phrases
201 Citations1992Didier Bourigault
The type of analysis used (surface grammatical analysis) is highlighted, as the methodological approach adopted to adapt the rules (experimental approach).
Distributional part-of-speech tagging
163 Citations1995Hinrich Schütze
This paper presents an algorithm for tagging words whose part-of-speech properties are unknown, which categorizes word tokens in context instead of word types.
The MIT Press eBooksDeterministic Part-of-Speech Tagging with Finite-State Transducers
136 Citations1997Emmanuel Roche, Yves Schabes
Disambiguation of super parts of speech (or supertags)
133 Citations1994Aravind K. Joshi, B. Srinivas
This work presents techniques for disambiguating supertags using local information such as lexical preference and local lexical dependencies, and the performance results for various models of supertag disambIGuation such as unigram, trigram and dependency-based models.
Does Baum-Welch re-estimation help taggers?
129 Citations1994David Elworthy
Two experiments designed to determine how much manual training information is needed for speech tagging by Hidden Markov Model suggest that initial biasing of either lexical or transition probabilities is essential to achieve a good accuracy and reveal three distinct patterns of Baum-Welch reestimation.
Part-of-speech induction from scratch
128 Citations1993Hinrich Schütze
A method for inducing the parts of speech of a language and part-of-speech labels for individual words from a large text corpus and classifies both ambiguous and unambiguous words correctly with high accuracy is presented.
Parsing a natural language using mutual information statistics
127 Citations1990David M. Magerman, Mitchell P. Marcus
The generalized mutual information statistic is derived, the parsing algorithm is described, and results and sample output from the parser are presented.
From Morpheme to Utterance
125 Citations1970Zellig S. Harris
Journal of the ACMA Computational Approach to Grammatical Coding of English Words
118 Citations1963Sheldon Klein, Robert F. Simmons
A computational grammar coder which has been completely programmed and is oper~tional on the IBM 7090 is described, part of a complete syntactic annlysis system for which it accomplishes word-class coding, using a computational approach rather than the usual method of dictionary lookup.
Automatic stochastic tagging of natural language texts
98 Citations1995Evangelos Dermatas, George Kokkinakis
Parsing, word associations and typical predicate-argument relations
91 Citations1989Kenneth Church, William A. Gale +2 more
This paper will attempt to show that many of these types of concerns can be addressed with syntactic methods (symbol pushing, and need not require explicit semantic interpretation), and it is possible to identify many of these interesting co-occurrence relations by computing simple summary statistics over millions of words of text.
Speech CommunicationSpeech recognition using hidden Markov models: A CMU perspective
87 Citations1990Kai-Fu Lee, Hsiao-Wuen Hon +2 more
This chapter introduces hidden Markov modeling techniques, analyze the reason for their success, and describe some improvements to the standard HMM used is SPHINX, a large-vocabulary, speaker-independent, continuous-speech recognition system developed at CMU.
Tagging and morphological disambiguation of Turkish text
80 Citations1994Kemal Oflazer, Ìlker Kuruöz
Preliminary results indicate that the tagger can tag about 98-99% of the texts accurately with very minimal user intervention, and an LFG parser developed for Turkish, generates, on the average, 50% less ambiguous parses and parses almost 2.5 times faster.
Finite-state parsing and disambiguation
78 Citations1990Kimmo Koskenniemi
A language-independent method of finite-state surface syntactic parsing and word-disambiguation is discussed, which excludes all ungrammatical possibilities leaving the correct interpretation of the sentence.
White Rose Research Online (University of Leeds, The University of Sheffield, University of York)The Automatic Grammatical Tagging of the LOB Corpus
74 Citations1983Geoffrey Leech, R. Garside +1 more
An account of the automatic grammatical tagging of the LOB (LancasterOslo/Bergen) Corpus of British English, with special reference to the methods of tagging the authors have adopted.
Weighted rational transductions and their application to human language processing
72 Citations1994Fernando Pereira, Michael Riley +1 more
The concepts of weighted language, transduction and automaton from algebraic automata theory are presented as a general framework for describing and implementing decoding cascades in speech and language processing and applications to speech recognition and Chinese text segmentation are discussed.
arXiv (Cornell University)NPtool, a detector of English noun phrases
70 Citations1995Atro Voutilainen
A syntax-based part-of-speech analyser
70 Citations1995Atro Voutilainen
A new tagger of English that uses only linguistic distributional rules is outlined and empirically evaluated, and it is suggested that the feasibility of the linguistic approach also in part-of-speech analysis is suggested.
Edinburgh Research Archive (University of Edinburgh)Finding structure in language
62 Citations1995Steven Finch
This thesis investigates the hypothesis that simple statistics of a large, noisy, unlabelled corpus of natural language can be exploited to discover some of the structure which exists in natural language automatically.
Extracting noun phrases from large-scale texts
62 Citations1994Kuang‐Hua Chen, Hsin‐Hsi Chen
A probabilistic chunker is applied to deciding the implicit boundaries of constituents and utilize the linguistic knowledge to extract the noun phrases by a finite state mechanism.
SRI International
54 Citations1992Jerry R. Hobbs, Douglas E. Appelt +3 more
FASTUS is a system for extracting information from free text in English, and possibly other languages as well, for entry into a database, and potentially for other applications.
Decision tree models applied to the labeling of text with parts-of-speech
54 Citations1992Ezra Black, Fred Jelinek +3 more
This work describes work which uses decision trees to estimate marginal probabilities in a maximum entropy model for predicting the part-of-speech of a word given the context in which it appears.
Rapid Incremental Parsing with Repair
50 Citations1990Steven Abney
This work describes a method of achieving rapid, reliable parsing of natural text through the application of three techniques: resolving small questions sequentially, repairing errors directly, instead of searching through a non-deterministic space, and recognizing major constituents before analyzing the details of their internal structure.
Compiling and using finite-state syntactic rules
48 Citations1992Kimmo Koskenniemi, Pasi Tapanainen +1 more
Fragments of a fairly comprehensive finite-state grammar of English are presented here, including samples from non-finite constructions as a demonstration of the capacity of the present formalism, which goes far beyond plain disambiguation or part of speech tagging.
Finding clauses in unrestricted text by finitary and stochastic methods
44 Citations1988Eva Ejerhed
A comparison of the error rates of the two parsing methods in the recognition of basic clauses showed that there was a 13% error rates for the regular expression method and a 6.5% error rate for the stochastic method.
Specifying a shallow grammatical representation for parsing purposes
30 Citations1995Atro Voutilainen, Timo Järvinen
This article reports on a double-blind experiment with a surface-oriented morphosyntactic grammatical representation used in a large-scale English parser and argues that a consistently applicable representation for morphology and also shallow syntax can be specified.
Journal of the American Society for Information ScienceAutomatic syntactic analysis of free text
28 Citations1990Christoph Schwarz
The system called COPSY (context operator syntax), which uses natural language processing techniques during fully automatic syntactic analysis of free text documents is described, which is being tested by the U.S. Department of Commerce for patent search and indexing.
Exploring the Statistical Derivation of Transformational Rule Sequences for Part-of-Speech Tagging
26 Citations1994Lance Ramshaw, Mitchell P. Marcus
Partial parsing
21 Citations1991Ralph Weischedel, Damaris Ayuso +4 more
This paper reports a handful of experiments designed to test the feasibility of applying well-known partial parsing techniques to the problem of automatic data base update from an open-ended source of messages.
DSpace repository (University of Tartu)Tagging the Teleman Corpus
18 Citations1995Thorsten Brants, Christer Samuelsson
Parsec: a connectionist learning architecture for parsing spoken language
18 Citations1992Ajay N. Jain
The PARSEC architecture, its training algorithms, and detailed performance analyses along several dimensions that concretely demonstrate PARSEC's advantages are presented.
Text, speech and language technologyImplementation and Evaluation of a German HMM for POS Disambiguation
17 Citations1999Helmut Feldweg
A German language model for the Xerox HMM tagger is presented and its performance is compared with two other German taggers with partial parameter re-estimation and full adaption of parameters from pre-tagged corpora.
Language and SpeechA Probabilistic Procedure for Grouping Words into Phrases
15 Citations1965Walter Stolz
A procedure based on the frequency and redundancy of sequences of syntactic word-classes was devised to identify behavioural word groups, or phrases, and a small sample of these phrases, derived from processing a short corpus of running text was compared with phrases produced by immediate constituent analysis of the same text.
Research Commons (University of Waikato)Language inference from function words
13 Citations1993Tony C. Smith, Ian H. Witten
A system that uncovers and characterizes regularities through principled wholesale pattern analysis of copious amounts of machine-readable text and compares well with current linguistic theories of syntax and offers a wide range of potential applications.
arXiv (Cornell University)Development of a Spanish Version of the Xerox Tagger
11 Citations1995Fernando Sánchez León, Amalio Francisco Nieto Serrano
The issue of adapting the Xerox Tagger to Spanish in order to tag the Spanish version of the ITU (International Telecommunications Union) corpus is addressed.
arXiv (Cornell University)Different Issues in the Design of a Lemmatizer/Tagger for Basque
10 Citations1995Itziar Aduriz, Iñaki Alegria +6 more
This paper presents relevant issues that have been considered in the design of a general purpose lemmatizer/tagger for Basque (EUSLEM) and on the strategies for morphological disambiguation that will be used.
Learning structural models of subword units through grammatical inference techniques
9 Citations1991Ernesto Julià Sanchís, Francisco Casacuberta +2 more
The authors propose obtaining the structure of phonetic units from training samples of speech automatically by using two specific grammatical inference algorithms: the error correcting GI algorithm and the morphic generator GI (MGGI) methodology.
Studies in natural language and linguistic theorySyntactic Affixation and Performance Structures
7 Citations1991Steven Abney
It is shown that functional elements also combine with their associated thematic elements to form word-like units I call chunks, for example, the segment of a noun phrase from the determiner to the head noun is a chunk.