Gated End-to-End Memory Networks
Published 1 January 2017Open access
Fei Liu, Julien Pérez
Citations100
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A novel end-to-end memory access regulation mechanism inspired by the current progress on the connection short-cutting principle in the field of computer vision is introduced, which is the first of its kind in the world.
Abstract
Machine reading using differentiable reasoning models has recently shown remarkable progress.
Keywords
Computer ScienceEngineering
Deep Residual Learning for Image Recognition
222,082 Citations2016Kaiming He, Xiangyu Zhang +2 more
This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.
Neural ComputationLong Short-Term Memory
98,079 Citations1997Sepp Hochreiter, Jürgen Schmidhuber
A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
Journal of Machine Learning ResearchVisualizing Data using t-SNE
35,654 Citations2008Laurens van der Maaten, Geoffrey E. Hinton
A new technique called t-SNE that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map, a variation of Stochastic Neighbor Embedding that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map.
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
18,777 Citations2015Kaiming He, Xiangyu Zhang +2 more
This work proposes a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit and derives a robust initialization method that particularly considers the rectifier nonlinearities.
Choice Reviews OnlineNeural networks for pattern recognition
18,687 Citations1994
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Understanding the difficulty of training deep feedforward neural networks
12,692 Citations2010Xavier Glorot, Yoshua Bengio
The objective here is to understand better why standard gradient descent from random initialization is doing so poorly with deep neural networks, to better understand these recent relative successes and help design better algorithms in the future.
Neural Networks for Pattern Recognition
12,198 Citations1995Chris Bishop
Cambridge University Press eBooksPattern Recognition and Neural Networks
6,468 Citations1996B. D. Ripley
Oxford University Research Archive (ORA) (University of Oxford)Teaching Machines to Read and Comprehend
1,936 Citations2015Karl Moritz Hermann, Tomáš Kočiský +5 more
arXiv (Cornell University)Teaching Machines to Read and Comprehend
1,526 Citations2015Karl Moritz Hermann, Tomáš Kočiský +5 more
A new methodology is defined that resolves this bottleneck and provides large scale supervised reading comprehension data that allows a class of attention based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure to be developed.
arXiv (Cornell University)Training Very Deep Networks
1,100 Citations2015Rupesh K. Srivastava, Klaus Greff +1 more
arXiv (Cornell University)Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
1,014 Citations2013Andrew Saxe, James L. McClelland +1 more
arXiv (Cornell University)Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
727 Citations2015Jason Weston, Antoine Bordes +5 more
arXiv (Cornell University)End-To-End Memory Networks
689 Citations2015Sainbayar Sukhbaatar, Arthur Szlam +2 more
A neural network with a recurrent attention model over a possibly large external memory that is trained end-to-end, and hence requires significantly less supervision during training, making it more generally applicable in realistic settings.
MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
668 Citations2013Matthew Richardson, Christopher J. C. Burges +1 more
MCTest is presented, a freely available set of stories and associated questions intended for research on the machine comprehension of text that requires machines to answer multiple-choice reading comprehension questions about fictional stories, directly tackling the high-level goal of open-domain machine comprehension.
arXiv (Cornell University)Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
631 Citations2015Ankit Kumar, Ozan İrsoy +7 more
The dynamic memory network (DMN), a neural network architecture which processes input sequences and questions, forms episodic memories, and generates relevant answers, is introduced.
arXiv (Cornell University)Dynamic Memory Networks for Visual and Textual Question Answering
595 Citations2016Caiming Xiong, Stephen Merity +1 more
The new DMN+ model improves the state of the art on both the Visual Question Answering dataset and the \babi-10k text question-answering dataset without supporting fact supervision.
The Second Dialog State Tracking Challenge
580 Citations2014Matthew Henderson, Blaise Thomson +1 more
The results suggest that while large improvements on a competitive baseline are possible, trackers are still prone to degradation in mismatched conditions and ensemble learning demonstrates the most accurate tracking can be achieved by combining multiple trackers.
arXiv (Cornell University)Training Very Deep Networks
546 Citations2015Rupesh K. Srivastava, Klaus Greff +1 more
A new architecture designed to overcome the challenges of training very deep networks, inspired by Long Short-Term Memory recurrent networks, which allows unimpeded information flow across many layers on information highways.
A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
442 Citations2016Danqi Chen, Jason Bolton +1 more
A thorough examination of this new reading comprehension task by creating over a million training examples by pairing CNN and Daily Mail news articles with their summarized bullet points, and showing that a neural network can be trained to give good performance on this task.
International Conference on Learning RepresentationsExact solutions to the nonlinear dynamics of learning in deep linear neural networks
347 Citations2014Andrew Saxe, James L. McClelland +1 more
It is shown that deep linear networks exhibit nonlinear learning phenomena similar to those seen in simulations of nonlinear networks, including long plateaus followed by rapid transitions to lower error solutions, and faster convergence from greedy unsupervised pretraining initial conditions than from random initial conditions.
arXiv (Cornell University)The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
334 Citations2015Felix Hill, Antoine Bordes +2 more
arXiv (Cornell University)The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
307 Citations2016Felix Hill, Antoine Bordes +2 more
There is a sweet-spot, not too big and not too small, between single words and full sentences that allows the most meaningful information in a text to be effectively retained and recalled, and models which store explicit representations of long-term contexts outperform state-of-the-art neural language models at predicting semantic content words.
arXiv (Cornell University)Highway Networks
247 Citations2015Rupesh K. Srivastava, Klaus Greff +1 more
The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems
227 Citations2015Ryan Lowe, Nissan Pow +2 more
The Ubuntu Dialogue Corpus is introduced, a dataset containing almost 1 million multi-turn dialogues, with a total of over 7 million utterances and 100 million words, that provides a unique resource for research into building dialogue managers based on neural language models that can make use of large amounts of unlabeled data.
International Conference on Machine LearningDynamic memory networks for visual and textual question answering
224 Citations2016Caiming Xiong, Stephen Merity +1 more
International Conference on Learning RepresentationsTowards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
188 Citations2016Jason Weston, Antoine Bordes +5 more
This work argues for the usefulness of a set of proxy tasks that evaluate reading comprehension via question answering, and classify these tasks into skill sets so that researchers can identify (and then rectify) the failings of their systems.