A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering
Published 1 January 2015Open access
Di Wang, Eric Nyberg
Citations372
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The proposed method uses a stacked bidirectional Long-Short Term Memory network to sequentially read words from question and answer sentences, and then outputs their relevance scores, which outperforms previous work which requires syntactic features and external knowledge resources.
Abstract
Di Wang, Eric Nyberg. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 2015.
Keywords
Computer Science
Neural ComputationLong Short-Term Memory
98,079 Citations1997Sepp Hochreiter, Jürgen Schmidhuber
A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
The Annals of StatisticsGreedy function approximation: A gradient boosting machine.
28,973 Citations2001Jerome H. Friedman
A general gradient descent boosting paradigm is developed for additive expansions based on any fitting criterion, and specific algorithms are presented for least-squares, least absolute deviation, and Huber-M loss functions for regression, and multiclass logistic likelihood for classification.
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
24,447 Citations2014Kyunghyun Cho, Bart van Merriënboer +5 more
Qualitatively, the proposed RNN Encoder‐Decoder model learns a semantically and syntactically meaningful representation of linguistic phrases.
arXiv (Cornell University)Distributed Representations of Words and Phrases and their Compositionality
18,086 Citations2013Tomáš Mikolov, Ilya Sutskever +3 more
This paper presents a simple method for finding phrases in text, and shows that learning good vector representations for millions of phrases is possible and describes a simple alternative to the hierarchical softmax called negative sampling.
arXiv (Cornell University)Sequence to Sequence Learning with Neural Networks
13,362 Citations2014Ilya Sutskever, Oriol Vinyals +1 more
Speech recognition with deep recurrent neural networks
8,837 Citations2013Alex Graves, Abdelrahman Mohamed +1 more
This paper investigates deep recurrent neural networks, which combine the multiple levels of representation that have proved so effective in deep networks with the flexible use of long range context that empowers RNNs.
Leibniz-Zentrum für Informatik (Schloss Dagstuhl)LOL: An Investigation into Cybernetic Humor, or: Can Machines Laugh?
3,084 Citations2016Alex Graves, Gervasi, Vincenzo +1 more
This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time.
Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection
810 Citations2011Richard Socher, Eric Huang +3 more
This work introduces a method for paraphrase detection based on recursive autoencoders (RAE) and unsupervised RAEs based on a novel unfolding objective and learns feature vectors for phrases in syntactic trees to measure word- and phrase-wise similarity between two sentences.
Question Answering with Subgraph Embeddings
640 Citations2014Antoine Bordes, Sumit Chopra +1 more
A system which learns to answer questions on a broad range of topics from a knowledge base using few hand-crafted features, using low-dimensional embeddings of words and knowledge base constituents to score natural language questions against candidate answers.
IBM Journal of Research and DevelopmentIntroduction to “This is Watson”
473 Citations2012David Ferrucci
A brief history of the events and ideas that positioned the team to take on the Jeopardy! challenge, build Watson, IBM Watson™, and ultimately triumph is provided, and how the system performed at champion levels is summarized.
Semantic Parsing for Single-Relation Question Answering
423 Citations2014Wen-tau Yih, Xiaodong He +1 more
A semantic parsing framework based on semantic similarity for open domain question answering (QA) that achieves higher precision across different recall points compared to the previous approach, and can improve F1 by 7 points.
What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA
400 Citations2007Mengqiu Wang, Noah A. Smith +1 more
A probabilistic quasi-synchronous grammar, inspired by one proposed for machine translation, and parameterized by mixtures of a robust nonlexical syntax/alignment model with a(n optional) lexical-semantics-driven log-linear model is proposed.
A Neural Network for Factoid Question Answering over Paragraphs
323 Citations2014Mohit Iyyer, Jordan Boyd‐Graber +3 more
This work introduces a recursive neural network model, qanta, that can reason over question text input by modeling textual compositionality and applies it to a dataset of questions from a trivia competition called quiz bowl.
Finding predominant word senses in untagged text
315 Citations2004Diana McCarthy, Rob Koeling +2 more
This work presents work on the use of a thesaurus acquired from raw textual corpora and the WordNet similarity package to find predominant noun senses automatically, and demonstrates that this method discovers appropriate predominant senses for words from two domain-specific corpora.
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions
247 Citations2018Michael Heilman, Noah A. Smith
A logistic regression model that uses 33 syntactic features of edit sequences to classify the sentence pairs and leads to competitive performance in recognizing textual entailment, paraphrase identification, and answer selection for question answering.
Question answering passage retrieval using dependency relations
228 Citations2005Hang Cui, Renxu Sun +3 more
This work presents two methods for learning relation mapping scores from past QA pairs: one based on mutual information and the other on expectation maximization, which significantly outperforms state-of-the-art density-based passage retrieval methods.
Question Answering Using Enhanced Lexical Semantic Models
228 Citations2013Wen-tau Yih, Ming‐Wei Chang +2 more
This work focuses on improving the performance using models of lexical semantic resources and shows that these systems can be consistently and significantly improved with rich lexical semantics information, regardless of the choice of learning algorithms.
Answer Extraction as Sequence Tagging with Tree Edit Distance
178 Citations2013Xuchen Yao, Benjamin Van Durme +2 more
A linear-chain Conditional Random Field based on pairs of questions and their possible answer sentences, learning the association between questions and answer types is constructed, casting answer extraction as an answer sequence tagging problem for the first time.
Foundations and Trends® in Information RetrievalOpen-Domain Question–Answering
156 Citations2007John Prager
Open-Domain Question Answering is an introduction to the field of Question Answering that describes the architecture of a QA system before analysing in detail some of the specific approaches that have been successfully deployed by academia and industry designing and building such systems.
Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering
151 Citations2010Mengqiu Wang, Christopher D. Manning
This work captures the alignment by using a novel probabilistic model that models tree-edit operations on dependency parse trees and treats alignments as structured latent variables, and offers a principled framework for incorporating complex linguistic features.
Recursive Deep Models for Discourse Parsing
149 Citations2014Jiwei Li, Rumeng Li +1 more
This paper proposes a recursive model for discourse parsing that jointly models distributed representations for clauses, sentences, and entire discourses that obtains comparable performance regarding standard discoursing parsing evaluations when compared against current state-of-art systems.
On relevance weights with little relevance information
144 Citations1997Stephen Robertson, Steve Walker
This research highlights the need to understand more fully the role of emotion in the development of Syetema, as well as the role that language and social media have in this process.
Automatic Feature Engineering for Answer Selection and Extraction
132 Citations2013Aliaksei Severyn, Alessandro Moschitti
The results show that the models greatly improve on the state of the art, e.g., up to 22% on F1 (relative improvement) for answer extraction, while using no additional resources and no manual feature engineering.
arXiv (Cornell University)Question Answering with Subgraph Embeddings
119 Citations2014Antoine Bordes, Sumit Chopra +1 more
Mapping Dependencies Trees: An Application to Question Answering
79 Citations2003Vasin Punyakanok, Dan Roth +1 more
An approach for answer selection in a free form question answering task is described, representing both questions and candidate passages using dependency trees, and incorporating semantic information such as named entities in this representation.
ACM SIGIR ForumOn relevance weights with little relevance information
28 Citations1997Stephen Robertson, Steve Walker