Finite precision error analysis of neural network hardware implementations

IEEE Transactions on ComputersPublished 1 March 1993

J.L. Holt, Jenq–Neng Hwang

DOI10.1109/12.210171

Source URL

Citations228

SJR quartileQ1

SJR score1.16

SNIP1.61

TL;DR

A theoretical analysis of error due to finite precision computation was undertaken to determine the necessary precision for successful forward retrieving and back-propagation learning in a multilayer perceptron.

Abstract

Through parallel processing, low precision fixed point hardware can be used to build a very high speed neural network computing engine where the low precision results in a drastic reduction in system cost. The reduced silicon area required to implement a single processing unit is taken advantage of by implementing multiple processing units on a single piece of silicon and operating them in parallel. The important question which arises is how much precision is required to implement neural network algorithms on this low precision hardware. A theoretical analysis of error due to finite precision computation was undertaken to determine the necessary precision for successful forward retrieving and back-propagation learning in a multilayer perceptron. This analysis can easily be further extended to provide a general finite precision analysis technique by which most neural network algorithms under any set of hardware constraints may be evaluated.>

Keywords

Computer ScienceEngineering

Journal of the American Statistical AssociationProbability, Random Variables, and Stochastic Processes.

16,350 Citations1984Julia Abrahams, A. Papoulis

View PDF

Learning Internal Representations by Error Propagation

16,235 Citations1985David E. Rumelhart, Geoffrey E. Hinton +1 more

The MIT Press eBooksParallel Distributed Processing

15,227 Citations1986David E. Rumelhart, James L. McClelland +1 more

A VLSI architecture for high-performance, low-cost, on-chip learning

211 Citations1990Dan Hammerstrom

Using state-of-the-art technology and innovative architectural techniques, the author's architecture approaches the speed and cost of analog systems while retaining much of the flexibility of large, general-purpose parallel machines.

Journal of Parallel and Distributed ComputingA unified systolic architecture for artificial neural networks

101 Citations1989Sun‐Yuan Kung, Jenq–Neng Hwang

A unified formulation is provided for both the retrieving and the learning phases of most ANNs, and on the basis of the formulation, a programmable ring systolic array is developed that maximizes the strength of VLSI in terms of intensive and pipelined computing and yet circumvents the limitation on communication.

Back propagation simulations using limited precision calculations

87 Citations2002J.L. Holt, Troy Baker

Simulation results indicate that integer computation works quite well for the back-propagation algorithm, and the limited precision integer simulations performed as well as the floating point simulations.

Finite precision error analysis of neural network electronic hardware implementations

61 Citations2002J.L. Holt, Jenq–Neng Hwang

The intent is to provide a general framework by which most neural network algorithms under any set of hardware constraints may be evaluated and be used as a guide to the details of hardware design and algorithm implementation.

IEEE Transactions on Acoustics Speech and Signal ProcessingA systolic neural network architecture for hidden Markov models

43 Citations1989Jenq–Neng Hwang, J.A. Vlontzos +1 more

A programmable systolic array is proposed which maximizes the power of VLSI implementations in terms of intensive and pipelined computing and yet circumvents the limitation on communication.

Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Recursive least-squares learning algorithms for neural networks</title>

10 Citations1990P.S. Lewis, Jenq N. Hwang

A pair of recursive least squares (ItLS) algorithms for online training of multilayer perceptrons which are a class of feedforward artificial neural networks incorporate second order information about the training error surface in order to achieve faster learning rates than are possible using first order gradient descent algorithms such as the generalized delta rule.

To compute numerically: Concepts and strategies (Little, Brown computer systems series)

8 Citations1983Stephen M. Pizer

The expression of the oncogenes c-fos, c-jun and c-pan-ras was analyzed at the protein level in newly diagnosed cases of acute lymphoblastic leukemia (ALL) in children to substantiate the significant correlation of c- fos-protein expression with the relapse rate.