A Deep Architecture for Semantic Parsing
Published 1 January 2014Open access
Edward Grefenstette, Phil Blunsom, Nando de Freitas, Karl Moritz Hermann
Citations54
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper presents a novel deep learning architecture which provides a semantic parsing system through the union of two neural models of language semantics, which allows for the generation of ontology-specific queries from natural language statements and questions without the need for parsing.
Abstract
Many successful approaches to semantic parsing build on top of the syntactic analysis of text, and make use of distributional representations or statistical models to match parses to ontology-specific queries. This paper presents a novel deep learning architecture which provides a semantic parsing system through the union of two neural models of language semantics. It allows for the generation of ontology-specific queries from natural language statements and questions without the need for parsing, which makes it especially suitable to grammatically malformed or syntactically atypical text, such as tweets, as well as permitting the development of semantic parsers for resourcepoor languages.
Keywords
Computer Science
Recurrent neural network based language model
5,349 Citations2010Tomáš Mikolov, Martin Karafiát +3 more
Results indicate that it is possible to obtain around 50% reduction of perplexity by using mixture of several RNN LMs, compared to a state of the art backoff language model.
arXiv (Cornell University)Natural Language Processing (almost) from Scratch
5,172 Citations2011Ronan Collobert, Jason Weston +4 more
Natural Language Processing (almost) from Scratch
3,987 Citations2011Ronan Collobert, Jason Weston +4 more
Word Representations: A Simple and General Method for Semi-Supervised Learning
1,944 Citations2010Joseph Turian, Lev-Arie Ratinov +1 more
This work evaluates Brown clusters, Collobert and Weston (2008) embeddings, and HLBL (Mnih & Hinton, 2009) embeds of words on both NER and chunking, and finds that each of the three word representations improves the accuracy of these baselines.
Recurrent Continuous Translation Models
1,345 Citations2013Nal Kalchbrenner, Phil Blunsom
A class of probabilistic continuous translation models called Recurrent Continuous Translation Models that are purely based on continuous representations for words, phrases and sentences and do not rely on alignments or phrasal translation units are introduced.
Semantic Compositionality through Recursive Matrix-Vector Spaces
1,297 Citations2012Richard Socher, Brody Huval +2 more
A recursive neural network model that learns compositional vector representations for phrases and sentences of arbitrary syntactic type and length and can learn the meaning of operators in propositional logic and natural language is introduced.
Generating Text with Recurrent Neural Networks
1,166 Citations2011Ilya Sutskever, James Martens +1 more
The power of RNNs trained with the new Hessian-Free optimizer by applying them to character-level language modeling tasks is demonstrated, and a new RNN variant that uses multiplicative connections which allow the current input character to determine the transition matrix from one hidden state vector to the next is introduced.
Learning to parse database queries using inductive logic programming
682 Citations1996John M. Zelle, Raymond J. Mooney
Experimental results with a complete database-query application for U.S. geography show that CHILL is able to learn parsers that outperform a preexisting, hand-crafted counterpart, and provide direct evidence of the utility of an empirical approach at the level of a complete natural language application.
CORE Scholar (Wright State University)Relation Extraction with Matrix Factorization and Universal Schemas
587 Citations2013Sebastian Riedel, Limin Yao +2 more
Three new graphical models for statistical language modelling
582 Citations2007Andriy Mnih, Geoffrey E. Hinton
It is shown how real-valued distributed representations for words can be learned at the same time as learning a large set of stochastic binary hidden features that are used to predict the distributed representation of the next word from previous distributed representations.
Computational LinguisticsLearning Dependency-Based Compositional Semantics
523 Citations2012Percy Liang, Michael I. Jordan +1 more
A new semantic formalism, dependency-based compositional semantics (DCS) is developed and a log-linear distribution over DCS logical forms is defined and it is shown that the system obtains comparable accuracies to even state-of-the-art systems that do require annotated logical forms.
Studies in fuzziness and soft computingNeural Probabilistic Language Models
491 Citations2006Yoshua Bengio, Holger Schwenk +3 more
This work proposes to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences, and incorporates this new language model into a state-of-the-art speech recognizer of conversational speech.
Transactions of the Association for Computational LinguisticsWeakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions
444 Citations2013Yoav Artzi, Luke Zettlemoyer
This paper shows semantic parsing can be used within a grounded CCG semantic parsing approach that learns a joint model of meaning and context for interpreting and executing natural language instructions, using various types of weak supervision.
Multilingual Models for Compositional Distributed Semantics
276 Citations2014Karl Moritz Hermann, Phil Blunsom
A novel technique for learning semantic representations, which extends the distributional hypothesis to multilingual data and joint-space embeddings and demonstrates that these representations are semantically plausible and can capture semantic relationships across languages without parallel data.
Large-scale Semantic Parsing via Schema Matching and Lexicon Extension
276 Citations2013Qingqing Cai, Alexander Yates
A semantic parser for Freebase is developed based on a reduction to standard supervised training algorithms, schema matching, and pattern learning that is capable of parsing questions with an F1 that improves by 0.42 over a purely-supervised learning algorithm.
Scaling Semantic Parsers with On-the-Fly Ontology Matching
236 Citations2013Tom Kwiatkowski, Eunsol Choi +2 more
A new semantic parsing approach that learns to resolve ontological mismatches, which is learned from question-answer pairs, uses a probabilistic CCG to build linguistically motivated logicalform meaning representations, and includes an ontology matching model that adapts the output logical forms for each target ontology.
Learning Discriminative Projections for Text Similarity Measures
223 Citations2011Wen-tau Yih, Kristina Toutanova +2 more
A novel discriminative training method that projects the raw term vectors into a common, low-dimensional vector space, which not only outperforms existing state-of-the-art approaches, but also achieves high accuracy at low dimensions and is thus more efficient.
arXiv (Cornell University)Recurrent Convolutional Neural Networks for Discourse Compositionality
192 Citations2013Nal Kalchbrenner, Phil Blunsom
The discourse model coupled to the sentence model obtains state of the art performance on a dialogue act classification experiment and is able to capture both the sequentiality of sentences and the interaction between different speakers.
arXiv (Cornell University)A Joint Model of Language and Perception for Grounded Attribute Learning
184 Citations2012Cynthia Matuszek, Nicholas FitzGerald +3 more
This work presents an approach for joint learning of language and perception models for grounded attribute induction, which includes a language model based on a probabilistic categorial grammar that enables the construction of compositional meaning representations.
Oxford University Research Archive (ORA) (University of Oxford)The Role of Syntax in Vector Space Models of Compositional Semantics
158 Citations2013Karl Moritz Hermann, Phil Blunsom
This model leverages the CCG combinatory operators to guide a non-linear transformation of meaning within a sentence and is used to learn high dimensional embeddings for sentences and evaluate them in a range of tasks, demonstrating that the incorporation of syntax allows a concise model to learn representations that are both effective and general.
Studies in fuzziness and soft computingNeural Probabilistic Language Models
130 Citations2006Yoshua Bengio, Holger Schwenk +3 more
Semantic Frame Identification with Distributed Word Representations
93 Citations2014Karl Moritz Hermann, Dipanjan Das +2 more
arXiv (Cornell University)Multilingual Distributed Representations without Word Alignment
68 Citations2013Karl Moritz Hermann, Phil Blunsom
arXiv (Cornell University)Learning Multilingual Word Representations using a Bag-of-Words Autoencoder
41 Citations2014Stanislas Lauly, Alex Boulanger +1 more
This workshop paper investigates an autoencoder model for learning multilingual word representations that does without word-level alignements, and observes that this method compares favorably with a previously proposed method that exploits word- level alignments to learn word representations.