Bounds on rates of variable-basis and neural-network approximation
IEEE Transactions on Information TheoryPublished 1 January 2001
Věra Kůrková, Marcello Sanguineti
Citations105
SJR quartileQ1
SJR score1.46
SNIP1.76
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The tightness of bounds on rates of approximation by feedforward neural networks is investigated in a more general context of nonlinear approximation by variable-basis functions.
Abstract
The tightness of bounds on rates of approximation by feedforward neural networks is investigated in a more general context of nonlinear approximation by variable-basis functions. Tight bounds on the worst case error in approximation by linear combinations of n elements of an orthonormal variable basis are derived.
Keywords
Computer ScienceEngineeringPhysics and Astronomy
IEEE Transactions on Information TheoryUniversal approximation bounds for superpositions of a sigmoidal function
2,849 Citations1993Andrew R. Barron
The approximation rate and the parsimony of the parameterization of the networks are shown to be advantageous in high-dimensional settings and the integrated squared approximation error cannot be made smaller than order 1/n/sup 2/d/ uniformly for functions satisfying the same smoothness assumption.
Parallel Networks that Learn to Pronounce English Text
1,556 Citations1989Terrence J. Sejnowski
American Mathematical MonthlyIntroductory Real Analysis.
1,418 Citations1971J. M. H. Olmsted, A. N. Kolmogorov +2 more
Neural ComputationRegularization Theory and Neural Networks Architectures
1,355 Citations1995Federico Girosi, Michael Jones +1 more
This paper shows that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models and some of the neural networks, and introduces new classes of smoothness functionals that lead to different classes of basis functions.
The Annals of StatisticsA Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training
474 Citations1992Lee Jones
A general convergence criterion for certain iterative sequences in Hilbert space is presented and results establish an 0(1/ F4n) nonsampling convergence rate for projection pursuit regression and neural network training.
IEEE Transactions on Information TheoryComparison of worst case errors in linear and neural network approximation
141 Citations2002Věra Kůrková, Marcello Sanguineti
A theoretical framework for describing sets of multivariable functions for which worst case errors in linear approximation are larger than those in approximation by neural networks is developed in the context of nonlinear approximation by fixed versus variable basis functions.
Journal of Optimization Theory and ApplicationsApproximating Networks and Extended Ritz Method for the Solution of Functional Optimization Problems
140 Citations2002R. Zoppoli, Marcello Sanguineti +1 more
The approximate method, proposed, minimize the cost functions of the resulting nonlinear programming problems include complex averaging operations, and is considered as an extension to the Ritz method, for which fixed basis functions are used.
Journal of Approximation TheoryRandom Approximants and Neural Networks
124 Citations1996Yuly Makovoz
It is shown that under some mild restrictions,?n(f, K)n?1/2, where? n(K)?0 asn?∞, this fact is used to estimate the errors of certain neural net approximations.
Journal of Approximation TheoryUniform Approximation by Neural Networks
113 Citations1998Yuly Makovoz
Journal of Fourier Analysis and ApplicationsNonlinear Approximation by Trigonometric Sums
92 Citations1995Ronald DeVore, Vladimir Temlyakov
Birkhäuser Boston eBooksDimension-Independent Rates of Approximation by Neural Networks
77 Citations1997Věra Kůrková
A new norm called variation with respect to a family of functions is introduced and its basic properties are derived and upper estimates for functions satisfying certain integral equations are given.
IBM Journal of Research and DevelopmentDimension-independent bounds on the degree of approximation by neural networks
71 Citations1994H. N. Mhaskar, Charles A. Micchelli
Enough conditions are studied in order that a neural network having a single hidden layer consisting of n neurons, each with an activation function φ, can be constructed so as to give a mean square approximation to f within a given accuracy, independent of the number of variables.
Rate of approximation results motivated by robust neural network learning
61 Citations1993Christian J. Darken, Michael J. Donahue +2 more
Borders on the rate of approximation valid for Hilbert spaces are derived and bounds for L spaces, 1 < p < m, are derived, re-covering the 0(1 /&) bounds of Barron and Jones for the case p = 2.
Neural NetworksRepresentations and rates of approximation of real-valued Boolean functions by neural networks
59 Citations1998Věra Kůrková, Petr Savický +1 more
Upper bounds on rates of approximation of real-valued functions of d Boolean variables by one-hidden-layer perceptron networks are given and sets of functions where these norms grow either polynomially or exponentially with d are described.
IEEE Transactions on Neural NetworksNeural approximations for infinite-horizon optimal control of nonlinear stochastic systems
51 Citations1998Thomas Parisini, R. Zoppoli
Simulation results show that the proposed feedback control law may constitute an effective tool for solving a wide class of control problems traditionally regarded as difficult ones, to a sufficient degree of accuracy.
Journal of Computer and System SciencesApproximation and Learning of Convex Superpositions
32 Citations1997Leonid Gurvits, Pascal Koiran
A fairly general method for constructing classes of functions of finite scale-sensitive dimension, which includes the so-called Γ class of Barron, which was shown to satisfy a number of interesting approximation and closure properties.
Analysis MathematicaApproximation of functions of several variables by trigonometric polynomials with given number of harmonics, and estimates of ε-entropy
22 Citations1989É. S. Belinskii
Circuits Systems and Signal ProcessingA note on error bounds for approximation in inner product spaces
14 Citations1996Ajit Dingankar, Irwin W. Sandberg
An algorithm is given that generates simpler approximants to a general element in the closure of the convex hull of a subset of an inner product space with somewhat less computational cost.
IEEE Transactions on Automatic ControlThe unreasonable effectiveness of neural network approximation
5 Citations1999Ajit Dingankar
Birkhäuser Boston eBooksConstructive Function Approximation: Theory and Practice
5 Citations1997Domingo Docampo, D. Hush +1 more
The theoretical limits of finite constructive convex approximations of a given function in a Hilbert space using elements taken from a reduced subset are studied and the trade-off between the global error and the partial error is investigated.
Artificial Neural Nets and Genetic AlgorithmsTightness of Upper Bounds on Rates of Neural-Network Approximation
3 Citations2001Věra Kůrková, Marcello Sanguineti
Tightness of upper bounds on neural network approximation is investigated in the framework of variable-basis approximation and conditions are satisfied by Lipschitz sigmoidal perceptrons.