Evaluation of semantic role labeling and dependency parsing of automatic speech recognition output
Published 1 March 2010
Benoît Favre, Bernd Bohnet, Dilek Hakkani‐Tür
Citations8
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work extends the standard evaluation metrics for joint dependency parsing and SRL of text in order to be able to handle speech recognition output with word errors and sentence segmentation errors.
Abstract
Semantic role labeling (SRL) is an important module of spoken language understanding systems. This work extends the standard evaluation metrics for joint dependency parsing and SRL of text in order to be able to handle speech recognition output with word errors and sentence segmentation errors. We propose metrics based on word alignments and bags of relations, and compare their results on the output of several SRL systems on broadcast news and conversations of the OntoNotes corpus. We evaluate and analyze the relation between the performance of the subtasks that lead to SRL, including ASR, part-of-speech tagging or sentence segmentation. The tools are made available to the community.
Keywords
Computer Science
The Berkeley FrameNet Project
2,564 Citations1998Collin F. Baker, Charles J. Fillmore +1 more
This report will present the project's goals and workflow, and information about the computational tools that have been adapted or created in-house for this work.
Computational LinguisticsAutomatic Labeling of Semantic Roles
1,647 Citations2002Daniel Gildea, Daniel Jurafsky
Improved Inference for Unlexicalized Parsing
610 Citations2007Slav Petrov, Dan Klein
A novel coarse-to-fine method in which a grammar’s own hierarchical projections are used for incremental pruning, including a method for efficiently computing projections of a grammar without a treebank is presented.
Language Resources and EvaluationFrom treebank to propbank
538 Citations2002Paul Kingsbury, Martha Palmer
This paper describes the approach to the development of a Proposition Bank, which involves the addition of semantic information to the Penn English Treebank, and introduces metaframes as a technique for handling similar frames among near− synonymous verbs.
Shallow Semantic Parsing using Support Vector Machines.
368 Citations2004Sameer Pradhan, Wayne Ward +3 more
A machine learning al- algorithm for shallow semantic parsing based on Support Vector Machines which shows performance improvements through a number of new features and their ability to generalize to a new test set drawn from the AQUAINT corpus is measured.
The CoNLL-2009 shared task
339 Citations2009Jan Hajič, Jan Štěpánek +12 more
This shared task combines the shared tasks of the previous five years under a unique dependency-based formalism similar to the 2008 task and describes how the data sets were created and show their quantitative properties.
The NomBank Project: An Interim Report
315 Citations2004Adam Meyers, Ruth Reeves +5 more
The NomBank project is described, a project that will provide argument structure for instances of common nouns in the Penn Treebank II corpus, including its speci(cid:2)cations and the process involved in creating the resource.
DSpace repository (University of Tartu)Extended constituent-to-dependency conversion for English
280 Citations2007Richard Johansson, Pierre Nugues
A new method to convert English constituent trees using the Penn Treebank annotation style into dependency trees was described, which used a richer set of edge labels and introduced links to handle long-distance phenomena such as wh-movement and topicalization.
Multilingual semantic role labeling
209 Citations2009Anders Björkelund, Love Hafdell +1 more
This paper describes the contribution of a pipeline of independent, local classifiers that identify the predicate sense, the arguments of the predicates, and the argument labels to the semantic role labeling task of the CoNLL-2009 shared task in the closed challenge.
IEEE Signal Processing MagazineSpoken language understanding
139 Citations2008Renato De Mori, F. Béchet +4 more
Spoken language understanding and natural language understanding share the goal of obtaining a conceptual representation of natural language sentences and computational semantics performs a conceptualization of the world using computational processes for composing a meaning representation structure from available signs.
IEEE Transactions on Audio Speech and Language ProcessingThe AT&T spoken language understanding system
112 Citations2005N. K. Gupta, Gökhan Tür +4 more
The focus of this work is to exploit data and to use machine learning techniques to create scalable SLU systems which can be quickly deployed for new domains with minimal human intervention.
Dependency-based semantic role labeling of PropBank
105 Citations2008Richard Johansson, Pierre Nugues
This work presents a PropBank semantic role labeling system for English that is integrated with a dependency parser and is the first dependency-based semantic role labeler for PropBank that rivals constituent-based systems in terms of performance.
Semantic role labeling using dependency trees
99 Citations2004Kadri Hacıoğlu
A novel semantic role labeler based on dependency trees is developed by formulating the semantic role labeling as a classification problem of dependency relations into one of several semantic roles.
Semantic role chunking combining complementary syntactic views
73 Citations2005Sameer Pradhan, Kadri Hacıoğlu +3 more
This paper describes a semantic role labeling system that uses features derived from different syntactic views, and combines them within a phrase-based chunking paradigm, and gives a significant improvement in performance over roles produced by using any one of the syntactic view individually.
Efficient parsing of syntactic and semantic dependency structures
50 Citations2009Bernd Bohnet
The system combines and implements efficient parsing techniques to get a high accuracy as well as very good parsing and training time and reached the fourth place (ranked by the macro F1 score) in the joint task for syntactic and semantic dependency parsing.
SRI'S 2004 BROADCAST NEWS SPEECH TO TEXT SYSTEM
11 Citations2004Anand Venkataraman, Raghudeep Gadde +4 more
The system that was used to automatically transcribeBroadcastNewsSpeech in the 2004 NIST evaluations sponsored by the DARPA EARS program is described and results are presented, discussing whether or not increases in amounts of training data proportionately benefit the research community.
Robust dependency parsing for spoken language understanding of spontaneous speech
10 Citations2009Frédéric Béchet, Alexis Nasr
A syntactic parser for spontaneous speech geared towards the identification of verbal subcategorization frames and a reranker which is specially trained for a given application is described.
Semantic annotations for conversational speech: From speech transcriptions to predicate argument structures
3 Citations2008Arianna Bisazza, Marco Dinarelli +4 more
The semantic content, which can be automatically generated, for the design of advanced dialog systems based on machine learning approaches, is described and the proposed PAS and taxonomy of dialog acts appear to be promising for theDesign of more complex dialog systems.