Not Just a Black Box: Learning Important Features Through Propagating Activation Differences

arXiv (Cornell University)Published 5 May 2016Open access

Avanti Shrikumar, Peyton Greenside, А.В. Щербина, Anshul Kundaje

DOI10.48550/arxiv.1605.01713

Citations547

TL;DR

DeepLIFT (Learning Important FeaTures), an efficient and effective method for computing importance scores in a neural network that compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference.

Abstract

Note: This paper describes an older version of DeepLIFT. See https://arxiv.org/abs/1704.02685 for the newer version. Original abstract follows: The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Learning Important FeaTures), an efficient and effective method for computing importance scores in a neural network. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. We apply DeepLIFT to models trained on natural images and genomic data, and show significant advantages over gradient-based methods.

Keywords

Computer Science

Neural ComputationLong Short-Term Memory

98,079 Citations1997Sepp Hochreiter, Jürgen Schmidhuber

A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.

Lecture notes in computer scienceVisualizing and Understanding Convolutional Networks

15,444 Citations2014Matthew D. Zeiler, Rob Fergus

A novel visualization technique is introduced that gives insight into the function of intermediate feature layers and the operation of the classifier in large Convolutional Network models, used in a diagnostic role to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark.

View PDF

arXiv (Cornell University)Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

10,728 Citations2014Jun‐Young Chung, Çaǧlar Gülçehre +2 more

These advanced recurrent units that implement a gating mechanism, such as a long short-term memory (LSTM) unit and a recently proposed gated recurrent unit (GRU), are found to be comparable to LSTM.

View PDF

arXiv (Cornell University)Deep Inside Convolutional Networks: Visualising Image Classification\n Models and Saliency Maps

4,930 Citations2013Karen Simonyan, Andrea Vedaldi +1 more

View PDF

PLoS ONEOn Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

4,411 Citations2015Sebastian Bach, Alexander Binder +4 more

This work proposes a general solution to the problem of understanding classification decisions by pixel-wise decomposition of nonlinear classifiers by introducing a methodology that allows to visualize the contributions of single pixels to predictions for kernel-based classifiers over Bag of Words features and for multilayered neural networks.

View PDF

arXiv (Cornell University)Striving for Simplicity: The All Convolutional Net

2,603 Citations2014Jost Tobias Springenberg, Alexey Dosovitskiy +2 more

View PDF

Deconvolutional networks

1,688 Citations2010Matthew D. Zeiler, Dilip Krishnan +2 more

This work presents a learning framework where features that capture these mid-level cues spontaneously emerge from image data, based on the convolutional decomposition of images under a spar-sity constraint and is totally unsupervised.