Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoning
Published 1 January 2019Open access
Yichen Jiang, Mohit Bansal
Citations56
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work presents an interpretable, controller-based Self-Assembling Neural Modular Network for multi-hop reasoning, where four novel modules (Find, Relocate, Compare, NoOp) are designed to perform unique types of language reasoning.
Abstract
Yichen Jiang, Mohit Bansal. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Keywords
Computer Science
Neural ComputationLong Short-Term Memory
98,079 Citations1997Sepp Hochreiter, Jürgen Schmidhuber
A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
UvA-DARE (University of Amsterdam)Adam: A Method for Stochastic Optimization
84,783 Citations2014Diederik P. Kingma, Jimmy Ba
Glove: Global Vectors for Word Representation
33,769 Citations2014Jeffrey Pennington, Richard Socher +1 more
A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
32,525 Citations2019Jacob Devlin, Ming‐Wei Chang +2 more
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
Machine LearningSimple statistical gradient-following algorithms for connectionist reinforcement learning
7,386 Citations1992Ronald J. Williams
This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units that are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reInforcement tasks, and they do this without explicitly computing gradient estimates.
The Stanford CoreNLP Natural Language Processing Toolkit
7,238 Citations2014Christopher D. Manning, Mihai Surdeanu +4 more
The design and use of the Stanford CoreNLP toolkit is described, an extensible pipeline that provides core natural language analysis, and it is suggested that this follows from a simple, approachable design, straightforward interfaces, the inclusion of robust and good quality analysis components, and not requiring use of a large amount of associated baggage.
SQuAD: 100,000+ Questions for Machine Comprehension of Text
6,328 Citations2016Pranav Rajpurkar, Jian Zhang +2 more
A strong logistic regression model is built, which achieves an F1 score of 51.0%, a significant improvement over a simple baseline (20%).
arXiv (Cornell University)Neural Architecture Search with Reinforcement Learning
3,847 Citations2016Barret Zoph, Quoc V. Le
Know What You Don’t Know: Unanswerable Questions for SQuAD
2,164 Citations2018Pranav Rajpurkar, Robin Jia +1 more
SQuadRUn is a new dataset that combines the existing Stanford Question Answering Dataset (SQuAD) with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones.
Oxford University Research Archive (ORA) (University of Oxford)Teaching Machines to Read and Comprehend
1,936 Citations2015Karl Moritz Hermann, Tomáš Kočiský +5 more
arXiv (Cornell University)Teaching Machines to Read and Comprehend
1,526 Citations2015Karl Moritz Hermann, Tomáš Kočiský +5 more
A new methodology is defined that resolves this bottleneck and provides large scale supervised reading comprehension data that allows a class of attention based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure to be developed.
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
1,436 Citations2018Zhilin Yang, Peng Qi +5 more
It is shown that HotpotQA is challenging for the latest QA systems, and the supporting facts enable models to improve performance and make explainable predictions.
arXiv (Cornell University)DARTS: Differentiable Architecture Search
1,399 Citations2018Hanxiao Liu, Karen Simonyan +1 more
arXiv (Cornell University)Bidirectional Attention Flow for Machine Comprehension
1,290 Citations2016Min Joon Seo, Aniruddha Kembhavi +2 more
WikiQA: A Challenge Dataset for Open-Domain Question Answering
857 Citations2015Yi Yang, Wen-tau Yih +1 more
The WIKIQA dataset is described, a new publicly available set of question and sentence pairs, collected and annotated for research on open-domain question answering, which is more than an order of magnitude larger than the previous dataset.
arXiv (Cornell University)Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
727 Citations2015Jason Weston, Antoine Bordes +5 more
Zenodo (CERN European Organization for Nuclear Research)QAngaroo (MedHop + WikiHop) - Constructing Datasets for Multi-hop Reading Comprehension Across Documents
494 Citations2018Johannes Welbl, Pontus Stenetorp +1 more
Learning to Reason: End-to-End Module Networks for Visual Question Answering
492 Citations2017Ronghang Hu, Jacob Andreas +3 more
End-to-End Module Networks are proposed, which learn to reason by directly predicting instance-specific network layouts without the aid of a parser, and achieve an error reduction of nearly 50% relative to state-of-theart attentional approaches.
Learning to Compose Neural Networks for Question Answering
477 Citations2016Jacob Andreas, Marcus Rohrbach +2 more
A question answering model that applies to both images and structured knowledge bases that uses natural language strings to automatically assemble neural networks from a collection of composable modules that achieves state-of-the-art results on benchmark datasets.
arXiv (Cornell University)Dynamic Coattention Networks For Question Answering
337 Citations2016Caiming Xiong, Victor W. Zhong +1 more
arXiv (Cornell University)Dynamic Coattention Networks For Question Answering
315 Citations2016Caiming Xiong, Victor W. Zhong +1 more
Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning
204 Citations2018David Mascharka, Philip Tran +2 more
This paper proposes a set of visual-reasoning primitives which, when composed, manifest as a model capable of performing complex reasoning tasks in an explicitly-interpretable manner, and shows that these primitives are highly performant, achieving state-of-the-art accuracy on the CLEVR dataset.
International Conference on Learning RepresentationsTowards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
188 Citations2016Jason Weston, Antoine Bordes +5 more
This work argues for the usefulness of a set of proxy tasks that evaluate reading comprehension via question answering, and classify these tasks into skill sets so that researchers can identify (and then rectify) the failings of their systems.
Multi-hop Reading Comprehension through Question Decomposition and Rescoring
183 Citations2019Sewon Min, Victor W. Zhong +2 more
A system that decomposes a compositional question into simpler sub-questions that can be answered by off-the-shelf single-hop RC models is proposed and a new global rescoring approach is introduced that considers each decomposition to select the best final answer, greatly improving overall performance.
Lecture notes in computer scienceExplainable Neural Computation via Stack Neural Module Networks
183 Citations2018Ronghang Hu, Jacob Andreas +2 more
A novel neural modular approach that performs compositional reasoning by automatically inducing a desired sub-task decomposition without relying on strong supervision is presented, which is more interpretable to human evaluators compared to other state-of-the-art models.
Compositional Questions Do Not Necessitate Multi-hop Reasoning
129 Citations2019Sewon Min, Eric Wallace +4 more
This work introduces a single-hop BERT-based RC model that achieves 67 F1—comparable to state-of-the-art multi-hop models and designs an evaluation setting where humans are not shown all of the necessary paragraphs for the intendedmulti-hop reasoning but can still answer over 80% of questions.
Understanding Dataset Design Choices for Multi-hop Reasoning
93 Citations2019Jifan Chen, Greg Durrett
This paper investigates two recently proposed datasets, WikiHop and HotpotQA, and explores sentence-factored models for these tasks; by design, these models cannot do multi-hop reasoning, but they are still able to solve a large number of examples in both datasets.
Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA
82 Citations2019Yichen Jiang, Mohit Bansal
This paper shows that in the multi-hop HotpotQA dataset, the examples often contain reasoning shortcuts through which models can directly locate the answer by word-matching the question with a sentence in the context, and shows that the 2-hop model trained on the regular data is more robust to the adversaries than the baseline.
Proceedings of the AAAI Conference on Artificial IntelligenceUsing Syntax to Ground Referring Expressions in Natural Images
55 Citations2018Volkan Cirik, Taylor Berg-Kirkpatrick +1 more
GroundNet, a neural network for referring expression recognition---the task of localizing in an image the object referred to by a natural language expression, is introduced, the first to rely on a syntactic analysis of the input referring expression in order to inform the structure of the computation graph.
arXiv (Cornell University)Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering
47 Citations2019Victor W. Zhong, Caiming Xiong +2 more
The Coarse-grain Fine-grain Coattention Network (CFC), a new question answering model that combines information from evidence across multiple documents that obtains a new state-of-the-art result on the Qangaroo WikiHop multi-evidence question answering task.
International Conference on Learning RepresentationsCoarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering
36 Citations2019Victor W. Zhong, Caiming Xiong +2 more