Neural networks for localized approximation of real functions
Published 30 December 2002
H. N. Mhaskar
Citations25
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is demonstrated that, using standard ideas from the theory of spline approximation, it is possible to construct such networks to provide localized approximation and can be used for the simultaneous approximation of a function and its derivative.
Abstract
The problem of constructing universal networks capable of approximating all functions having bounded derivatives is discussed. It is demonstrated that, using standard ideas from the theory of spline approximation, it is possible to construct such networks to provide localized approximation. The networks can be used to implement multivariate analogues of the Chui-Wang wavelets (1990) and also for the simultaneous approximation of a function and its derivative. The number of neurons required to yield the desired approximation at any point does not depend upon the degree of accuracy desired.>
Keywords
Computer Science
Neural NetworksMultilayer feedforward networks are universal approximators
21,248 Citations1989Kurt Hornik, Maxwell B. Stinchcombe +1 more
It is rigorously established that standard multilayer feedforward networks with as few as one hidden layer using arbitrary squashing functions are capable of approximating any Borel measurable function from one finite dimensional space to another to any desired degree of accuracy, provided sufficiently many hidden units are available.
Mathematics of Control Signals and SystemsApproximation by superpositions of a sigmoidal function
13,684 Citations1989George Cybenko
Neural NetworksOn the approximate realization of continuous mappings by neural networks
4,220 Citations1989Ken-ichi Funahashi
It is proved that any continuous mapping can be approximately realized by Rumelhart-Hinton-Williams' multilayer neural networks with at least one hidden layer whose output functions are sigmoid functions.
IEEE Transactions on Information TheoryUniversal approximation bounds for superpositions of a sigmoidal function
2,849 Citations1993Andrew R. Barron
The approximation rate and the parsimony of the parameterization of the networks are shown to be advantageous in high-dimensional settings and the integrated squared approximation error cannot be made smaller than order 1/n/sup 2/d/ uniformly for functions satisfying the same smoothness assumption.
Mathematics of ComputationSpline Functions: Basic Theory.
2,493 Citations1982Philip W. Smith, Larry L. Schumaker
The material covered provides the reader with the necessary tools for understanding the many applications of splines in such diverse areas as approximation theory, computer-aided geometric design, curve and surface design and fitting, image processing, numerical solution of differential equations, and increasingly in business and the biosciences.
ScienceRegularization Algorithms for Learning That Are Equivalent to Multilayer Networks
1,042 Citations1990Tomaso Poggio, Federico Girosi
A theory is reported that shows the equivalence between regularization and a class of three-layer networks called regularization networks or hyper basis functions.
Bulletin of the London Mathematical SocietySPLINE FUNCTIONS: Basic Theory
549 Citations1982R. F. Churchhouse
Neural ComputationLocal Learning Algorithms
544 Citations1992Léon Bottou, Vladimir Vapnik
A single analysis suggests that neither kNN or RBF, nor nonlocal classifiers, achieve the best compromise between locality and capacity.
IEEE International Conference on Neural NetworksCapabilities of three-layered perceptrons
400 Citations1988Irie, Miyake
A theorem is proved that three-layered perceptrons with an infinite number of computing units can represent arbitrary mapping if the desired mapping and the input-output characteristics of the computing units satisfy some constraints.
Transactions of the American Mathematical SocietyOn compactly supported spline wavelets and a duality principle
400 Citations1992Charles K. Chui, Jianzhong Wang
Advances in Applied MathematicsApproximation by superposition of sigmoidal and radial basis functions
261 Citations1992H. N. Mhaskar, Charles A. Micchelli
Universal approximation using feedforward networks with non-sigmoid hidden layer activation functions
254 Citations1989Stinchcombe, White
Multilayer feedforward networks possess universal approximation capabilities by virtue of the presence of intermediate layers with sufficiently many parallel processors; the properties of the intermediate-layer activation function are not so crucial.
manuscripta mathematicaOptimal nonlinear approximation
232 Citations1989Ronald DeVore, Ralph Howard +1 more
Journal of Approximation TheoryApproximation by ridge functions and neural networks with one hidden layer
184 Citations1992Charles K. Chui, Xin Li
This configuration of an infinite set V of vectors in R s, s ⩾ 1, for which neural networks with one hidden layer of the form ∑ ( I, k) ϵ J c ( i, k ) σ (〈i,x〉+ k ), where k ϵ Z , c ( I , k )π� R , and i ϵZ s can be designed to approximate any continuous functions in s variables.
Advances in Computational MathematicsApproximation properties of a multilayered feedforward artificial neural network
151 Citations1993H. N. Mhaskar
It is proved that an artificial neural network with multiple hidden layers and akth-order sigmoidal response function can be used to approximate any continuous function on any compact subset of a Euclidean space so as to achieve the Jackson rate of approximation.
Approximating and learning unknown mappings using multilayer feedforward networks with bounded weights
108 Citations1990Maxwell B. Stinchcombe, Hannah J. White
It is shown that feedforward networks having bounded weights are not undesirable restricted, but are in fact universal approximators, provided that the hidden-layer activation function belongs to one of several suitable broad classes of functions: polygonal functions, certain piecewise polynomial functions, or a class of functions analytic on some open interval.
Mathematics of ComputationNeural networks for localized approximation
79 Citations1994Charles K. Chui, Xin Li +1 more