CFO: Conditional Focused Neural Question Answering with Large-scale Knowledge Bases
Published 1 January 2016Open access
Zihang Dai, Lei Li, Wei Xu
Citations126
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work proposes CFO, a Conditional Focused neural-network-based approach to answering factoid questions with knowledge bases that outperforms the current state of the art by an absolute margin of 11.8%.
Abstract
How can we enable computers to automatically answer questions like "Who created the character Harry Potter"? Carefully built knowledge bases provide rich sources of facts. However, it remains a challenge to answer factoid questions raised in natural language due to numerous expressions of one question. In particular, we focus on the most common questions -ones that can be answered with a single fact in the knowledge base. We propose CFO, a Conditional Focused neuralnetwork-based approach to answering factoid questions with knowledge bases. Our approach first zooms in a question to find more probable candidate subject mentions, and infers the final answers with a unified conditional probabilistic framework. Powered by deep recurrent neural networks and neural embeddings, our proposed CFO achieves an accuracy of 75.7% on a dataset of 108k questions -the largest public one to date. It outperforms the current state of the art by an absolute margin of 11.8%.
Keywords
Computer Science
Glove: Global Vectors for Word Representation
33,769 Citations2014Jeffrey Pennington, Richard Socher +1 more
A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
24,447 Citations2014Kyunghyun Cho, Bart van Merriënboer +5 more
Qualitatively, the proposed RNN Encoder‐Decoder model learns a semantically and syntactically meaningful representation of linguistic phrases.
Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
8,612 Citations2011John C. Duchi, Elad Hazan +1 more
A unified architecture for natural language processing
5,204 Citations2008Ronan Collobert, Jason Weston
This work describes a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense using a language model.
Translating embeddings for modeling multi-relational data
5,178 Citations2015Antoine Bordes, Nicolas Usunier +2 more
TransE is proposed, a method which models relationships by interpreting them as translations operating on the low-dimensional embeddings of the entities, which proves to be powerful since extensive experiments show that TransE significantly outperforms state-of-the-art methods in link prediction on two knowledge bases.
Freebase
4,892 Citations2008Kurt Bollacker, Colin Evans +3 more
MQL provides an easy-to-use object-oriented interface to the tuple data in Freebase and is designed to facilitate the creation of collaborative, Web-based data-oriented applications.
Leibniz-Zentrum für Informatik (Schloss Dagstuhl)NER in Archival Finding Aids
3,270 Citations2021Zhiheng Huang, Wei Xu +1 more
This work has created a set of datasets to train Machine Learning algorithms to find named entities and geographical locations and tested them in several datasets and registered their precision and accuracy.
Semantic WebDBpedia – A large-scale, multilingual knowledge base extracted from Wikipedia
3,149 Citations2015Jens Lehmann, Robert Isele +9 more
An overview of the DBpedia community project is given, including its architecture, technical implementation, maintenance, internationalisation, usage statistics and applications, including DBpedia one of the central interlinking hubs in the Linked Open Data (LOD) cloud.
Leibniz-Zentrum für Informatik (Schloss Dagstuhl)LOL: An Investigation into Cybernetic Humor, or: Can Machines Laugh?
3,084 Citations2016Alex Graves, Gervasi, Vincenzo +1 more
This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time.
Journal of Machine Learning ResearchA neural probabilistic language model
2,663 Citations2003BengioYoshua, DucharmeRéjean +2 more
This work reports on experiments using neural networks for the probability function, showing on two text corpora that the proposed approach very significantly improves on a state-of-the-art trigram model.
arXiv (Cornell University)Recurrent Neural Network Regularization
2,274 Citations2014Wojciech Zaremba, Ilya Sutskever +1 more
This paper shows how to correctly apply dropout to LSTMs, and shows that it substantially reduces overfitting on a variety of tasks.
Proceedings of the AAAI Conference on Artificial IntelligenceToward an Architecture for Never-Ending Language Learning
1,991 Citations2010J. Andrew Carlson, Justin Betteridge +4 more
Oxford University Research Archive (ORA) (University of Oxford)Teaching Machines to Read and Comprehend
1,936 Citations2015Karl Moritz Hermann, Tomáš Kočiský +5 more
Semantic Parsing on Freebase from Question-Answer Pairs
1,591 Citations2013Jonathan Berant, Andrew Chou +2 more
This paper trains a semantic parser that scales up to Freebase and outperforms their state-of-the-art parser on the dataset of Cai and Yates (2013), despite not having annotated logical forms.
arXiv (Cornell University)Teaching Machines to Read and Comprehend
1,526 Citations2015Karl Moritz Hermann, Tomáš Kočiský +5 more
A new methodology is defined that resolves this bottleneck and provides large scale supervised reading comprehension data that allows a class of attention based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure to be developed.
Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base
796 Citations2015Wen-tau Yih, Ming‐Wei Chang +2 more
This work proposes a novel semantic parsing framework for question answering using a knowledge base that leverages the knowledge base in an early stage to prune the search space and thus simplifies the semantic matching problem.
arXiv (Cornell University)Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
727 Citations2015Jason Weston, Antoine Bordes +5 more
Learning to parse database queries using inductive logic programming
682 Citations1996John M. Zelle, Raymond J. Mooney
Experimental results with a complete database-query application for U.S. geography show that CHILL is able to learn parsers that outperform a preexisting, hand-crafted counterpart, and provide direct evidence of the utility of an empirical approach at the level of a complete natural language application.
Question Answering with Subgraph Embeddings
640 Citations2014Antoine Bordes, Sumit Chopra +1 more
A system which learns to answer questions on a broad range of topics from a knowledge base using few hand-crafted features, using low-dimensional embeddings of words and knowledge base constituents to score natural language questions against candidate answers.
arXiv (Cornell University)Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
631 Citations2015Ankit Kumar, Ozan İrsoy +7 more
The dynamic memory network (DMN), a neural network architecture which processes input sequences and questions, forms episodic memories, and generates relevant answers, is introduced.
arXiv (Cornell University)Large-scale Simple Question Answering with Memory Networks
562 Citations2015Antoine Bordes, Nicolas Usunier +2 more
This paper studies the impact of multitask and transfer learning for simple question answering; a setting for which the reasoning required to answer is quite easy, as long as one can retrieve the correct evidence given a question, which can be difficult in large-scale conditions.
Semantic Parsing via Paraphrasing
535 Citations2014Jonathan Berant, Percy Liang
This paper presents two simple paraphrase models, an association model and a vector space model, and trains them jointly from question-answer pairs, improving state-of-the-art accuracies on two recently released question-answering datasets.
Computational LinguisticsLearning Dependency-Based Compositional Semantics
523 Citations2012Percy Liang, Michael I. Jordan +1 more
A new semantic formalism, dependency-based compositional semantics (DCS) is developed and a log-linear distribution over DCS logical forms is defined and it is shown that the system obtains comparable accuracies to even state-of-the-art systems that do require annotated logical forms.
Information Extraction over Structured Data: Question Answering with Freebase
508 Citations2014Xuchen Yao, Benjamin Van Durme
It is shown that relatively modest information extraction techniques, when paired with a webscale corpus, can outperform these sophisticated approaches by roughly 34% relative gain.
Semantic Parsing for Single-Relation Question Answering
423 Citations2014Wen-tau Yih, Xiaodong He +1 more
A semantic parsing framework based on semantic similarity for open domain question answering (QA) that achieves higher precision across different recall points compared to the previous approach, and can improve F1 by 7 points.
Paraphrase-Driven Learning for Open Question Answering
343 Citations2013Anthony Fader, Luke Zettlemoyer +1 more
This work demonstrates that it is possible to learn a semantic lexicon and linear ranking function without manually annotating questions and automatically generalizes a seed lexicon, and includes a scalable, parallelized perceptron parameter estimation scheme.
A Neural Network for Factoid Question Answering over Paragraphs
323 Citations2014Mohit Iyyer, Jordan Boyd‐Graber +3 more
This work introduces a recursive neural network model, qanta, that can reason over question text input by modeling textual compositionality and applies it to a dataset of questions from a trivia competition called quiz bowl.
Contrastive estimation
320 Citations2005Noah A. Smith, Jason Eisner
A novel approach, contrastive estimation, is described, which outperforms EM, is more robust to degradations of the dictionary, and can largely recover by modeling additional features.
Lecture notes in computer scienceOpen Question Answering with Weakly Supervised Embedding Models
301 Citations2014Antoine Bordes, Jason Weston +1 more
This paper empirically demonstrate that the model can capture meaningful signals from its noisy supervision leading to major improvements over paralex, the only existing method able to be trained on similar weakly labeled data.
Large-scale Semantic Parsing via Schema Matching and Lexicon Extension
276 Citations2013Qingqing Cai, Alexander Yates
A semantic parser for Freebase is developed based on a reduction to standard supervised training algorithms, schema matching, and pattern learning that is capable of parsing questions with an F1 that improves by 0.42 over a purely-supervised learning algorithm.
Traversing Knowledge Graphs in Vector Space
274 Citations2015Kelvin Guu, J. J. Miller +1 more
It is demonstrated that compositional training acts as a novel form of structural regularization, reliably improving performance across all base models (reducing errors by up to 43%) and achieving new state-of-the-art results.
Lecture notes in computer scienceUsing Multiple Clause Constructors in Inductive Logic Programming for Semantic Parsing
268 Citations2001Lappoon R. Tang, Raymond J. Mooney
Preliminary results demonstrated that an approach which combines different learning methods in inductive logic programming (ILP) to allow a learner to produce more expressive hypotheses than that of each individual learner is promising.
Transactions of the Association for Computational LinguisticsLarge-scale Semantic Parsing without Question-Answer Pairs
217 Citations2014Siva Reddy, Mirella Lapata +1 more
This paper introduces a novel semantic parsing approach to query Freebase in natural language without requiring manual annotations or question-answer pairs and converts sentences to semantic graphs using CCG and subsequently grounds them to Freebase guided by denotations as a form of weak supervision.
International Conference on Learning RepresentationsTowards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
188 Citations2016Jason Weston, Antoine Bordes +5 more
This work argues for the usefulness of a set of proxy tasks that evaluate reading comprehension via question answering, and classify these tasks into skill sets so that researchers can identify (and then rectify) the failings of their systems.
arXiv (Cornell University)Traversing Knowledge Graphs in Vector Space
99 Citations2015Kelvin Guu, J. J. Miller +1 more
Joint Relational Embeddings for Knowledge-based Question Answering
86 Citations2014Min-Chul Yang, Nan Duan +2 more
This paper proposes a novel embedding-based approach that maps NL-questions into LFs for KBQA by leveraging semantic associations between lexical representations and KBproperties in the latent space.
arXiv (Cornell University)Open Question Answering with Weakly Supervised Embedding Models
86 Citations2014Antoine Bordes, Jason Weston +1 more
Dropout Improves Recurrent Neural Networks for Handwriting Recognition
75 Citations2014Vu Pham, Théodore Bluche +2 more
It is shown that RNNs with Long Short-Term memory cells can be greatly improved using dropout - a recently proposed regularization method for deep architectures, even when the network mainly consists of recurrent and shared connections.
Transactions of the Association for Computational LinguisticsImitation Learning of Agenda-based Semantic Parsers
75 Citations2015Jonathan Berant, Percy Liang
This paper combines ideas from imitation learning and agenda-based parsing to train a semantic parser that searches partial logical forms in a more strategic order, and obtains a 6x-9x speedup over fixed-order parsing, while maintaining comparable accuracy.
arXiv (Cornell University)Towards Neural Network-based Reasoning
61 Citations2015Baolin Peng, Zhengdong Lu +2 more
The empirical studies show that Neural Reasoner can outperform existing neural reasoning systems with remarkable margins on two difficult artificial tasks (Positional Reasoning and Path Finding) proposed in [8].