A Fast, Compact Approximation of the Exponential Function

Neural ComputationPublished 1 May 1999

Nicol N. Schraudolph

DOI10.1162/089976699300016467

Source URL

Citations152

SJR quartileQ1

SJR score0.83

SNIP1.45

TL;DR

This article describes how exponentiation can be approximated by manipulating the components of a standard (IEEE-754) floating-point representation, which models the exponential function as a lookup table with linear interpolation, but is significantly faster and more compact.

Abstract

Neural network simulations often spend a large proportion of their time computing exponential functions. Since the exponentiation routines of typical math libraries are rather slow, their replacement with a fast approximation can greatly reduce the overall computation time. This article describes how exponentiation can be approximated by manipulating the components of a standard (IEEE-754) floating-point representation. This models the exponential function as well as a lookup table with linear interpolation, but is significantly faster and more compact.

Keywords

Computer SciencePhysics and Astronomy

Advances in Computational MathematicsOn the LambertW function

6,184 Citations1996Robert M. Corless, Gastón H. Gonnet +3 more

A new discussion of the complex branches of W, an asymptotic expansion valid for all branches, an efficient numerical procedure for evaluating the function to arbitrary precision, and a method for the symbolic integration of expressions containing W are presented.

Neural ComputationLearning to Forget: Continual Prediction with LSTM

5,419 Citations2000Felix A. Gers, Jürgen Schmidhuber +1 more

This work identifies a weakness of LSTM networks processing continual input streams that are not a priori segmented into subsequences with explicitly marked ends at which the network's internal state could be reset, and proposes a novel, adaptive forget gate that enables an LSTm cell to learn to reset itself at appropriate times, thus releasing internal resources.

Mathematics and Computers in SimulationAnalytical approximations for real values of the Lambert W-function

244 Citations2000D. A. Barry, J.‐Y. Parlange +4 more

View PDF

Communications of the ACMAlgorithm 443: Solution of the transcendental equation <i> we <sup>w</sup> = x </i>

117 Citations1973F.N. Fritsch, Robert E. Shafer +1 more

WEW solves the transcendental equation the authors ~ = x for w, given x > 0, by an iteration that converges much more rapidly than either Newton's method or fixed-point iteration.

View PDF

Neural ComputationOn a Fast, Compact Approximation of the Exponential Function

19 Citations2000Gavin C. Cawley

A recoding of this procedure that overcomes some of the limitations of the original macro at little or no additional computational expense is communicated.

Neural ComputationJPEG Quality Transcoding Using Neural Networks Trained with a Perceptual Error Measure

11 Citations1999John Lazzaro, John Wawrzynek

Technology for JQT design that takes a pattern recognition approach to the problem, using a database of images to train statistical models of the artifacts introduced through JPEG compression, and uses a model of human visual perception as an error measure.