Support Vector Regression for the simultaneous learning of a multivariate function and its derivatives
NeurocomputingPublished 23 August 2005
Marcelino Lázaro, Ignacio Santamarı́a, Fernando Pérez‐Cruz, Antonio Artés-Rodrı́guez
Citations41
SJR quartileQ1
SJR score1.47
SNIP1.94
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The proposed method shows that using the information about derivatives significantly improves the reconstruction of the function and is derived an iterative re-weighted least squares (IRWLS) procedure that works fast for moderate-size problems.
Abstract
In this paper, the problem of simultaneously approximating a function and its derivatives is formulated within the Support Vector Machine (SVM) framework. First, the problem is solved for a one-dimensional input space by using the ε-insensitive loss function and introducing additional constraints in the approximation of the derivative. Then, we extend the method to multi-dimensional input spaces by a multidimensional regression algorithm. In both cases, to optimize the regression estimation problem, we have derived an iterative re-weighted least squares (IRWLS) procedure that works fast for moderate-size problems. The proposed method shows that using the information about derivatives significantly improves the reconstruction of the function.
Keywords
Computer ScienceEngineering
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Cambridge University Press eBooksNumerical optimization
14,107 Citations2021W. John Braun, Duncan J. Murdoch
Numerical Optimization
9,247 Citations1999Jorge Nocedal, Stephen J. Wright
Neural NetworksUniversal approximation of an unknown mapping and its derivatives using multilayer feedforward networks
2,086 Citations1990Kurt Hornik, Maxwell B. Stinchcombe +1 more
A shoulder strap retainer having a base to be positioned on the exterior shoulder portion of a garment with securing means attached to the undersurface of the base for removably securing the base to the exterior shoulders portion of the garment.
NeurocomputingWeighted least squares support vector machines: robustness and sparse approximation
1,249 Citations2002Johan A. K. Suykens, Jos De Brabanter +2 more
The methods of this paper are illustrated for RBF kernels and demonstrate how to obtain robust estimates with selection of an appropriate number of hidden units, in the case of outliers or non-Gaussian error distributions with heavy tails.
Neural NetworksOn learning the derivatives of an unknown mapping with multilayer feedforward networks
312 Citations1992A. Ronald Gallant, Halbert White
It is shown that a net can be trained so that the map and its derivatives are learned, and least squares and similar estimates are strongly consistent in Sobolev norm provided the number of hidden units and the size of the training set increase together.
IEEE Transactions on Neural NetworksComparison of adaptive methods for function estimation from samples
185 Citations1996Vladimir Cherkassky, D. Gehring +1 more
A pragmatic framework for comparisons between various methods is described, and a detailed comparison study comprising several thousand individual experiments is presented, which provides some insights on applicability of various methods.
Lecture notes in computer scienceMulti-dimensional Function Approximation and Regression Estimation
107 Citations2002Fernando Pérez‐Cruz, Gustau Camps‐Valls +4 more
A multi-dimensional Support Vector Regressor (MSVR) that uses a cost function with a hyperspherical insensitive zone, capable of obtaining better predictions than using an SVM independently for each dimension.
NeurocomputingSimultaneous approximations of multivariate functions and their derivatives by neural networks with one hidden layer
102 Citations1996Xin Li
For any compact set K of Rs, and any multivariate function f defined on an open set containing K, a neural network with one hidden layer can be so constructed that f and all its existing continuous kth order partial derivatives, for k = (k1, …, ks) ϵ Z+s satisfying ∑i = 1ski ≤ n, can be simultaneously and uniformly approximated on K by the network.
Applied Mathematical ModellingApproximation of functions and their derivatives: A neural network implementation with applications
63 Citations1999T. Nguyen-Thien, T. Tran‐Cong
A neural network implementation for the numerical approximation of functions of several variables and their first and second order partial derivatives can result in improved numerical methods for solving partial differential equations by eliminating the need to discretise the volume of the analysis domain.
Zenodo (CERN European Organization for Nuclear Research)An Irwls Procedure For Sir
62 Citations2015Fernando Pérez‐Cruz, Ángel Navia Vázquez +2 more
A new algorithm to train Support Vector Machines is included, covering both the proposed approach instead of the quadratic programming part and the most advanced methods to deal with large training data sets.
Neural ComputationConvergence of the IRWLS Procedure to the Support Vector Machine Solution
48 Citations2004Fernando Pérez‐Cruz, Carlos Bousoño‐Calzón +1 more
An iterative reweighted least squares (IRWLS) procedure recently proposed is shown to converge to the support vector machine solution.
Signal ProcessingA regularized technique for the simultaneous reconstruction of a function and its derivatives with application to nonlinear transistor modeling
9 Citations2003Marcelino Lázaro, Ignacio Santamarı́a +3 more
A new regularized digital filtering technique for the simultaneous approximation of a function and its derivatives is proposed and a regularized functional is defined, which includes a term forcing the smoothness of the solution.