Incremental extreme learning machine with fully complex hidden nodes
NeurocomputingPublished 2 October 2007Open access
Guang-Bin Huang, Ming-Bin Li, Lei Chen, Chee‐Kheong Siew
Citations255
SJR quartileQ4
SJR score0.11
SNIP0.06
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is shown that, as long as the hidden layer activation function is complex continuous discriminatory or complex bounded nonlinear piecewise continuous, I-ELM can still approximate any target functions in the complex domain.
Abstract
10.1016/j.neucom.2007.07.025
Keywords
Computer ScienceEngineering
NeurocomputingExtreme learning machine: Theory and applications
13,180 Citations2006Guang-Bin Huang, Qinyu Zhu +1 more
A new learning algorithm called ELM is proposed for feedforward neural networks (SLFNs) which randomly chooses hidden nodes and analytically determines the output weights of SLFNs which tends to provide good generalization performance at extremely fast learning speed.
Extreme learning machine: a new learning scheme of feedforward neural networks
4,112 Citations2005Guang-Bin Huang, Qin‐Yu Zhu +1 more
A new learning algorithm called extreme learning machine (ELM) for single-hidden layer feedforward neural networks (SLFNs) which randomly chooses the input weights and analytically determines the output weights of SLFNs is proposed.
Neural ComputationUniversal Approximation Using Radial-Basis-Function Networks
4,077 Citations1991Jihun Park, Irwin W. Sandberg
It is proved thatRBF networks having one hidden layer are capable of universal approximation, and a certain class of RBF networks with the same smoothing factor in each kernel node is broad enough for universal approximation.
Complex SystemsRadial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks
3,439 Citations1988David S. Broomhead, David Lowe
The relationship between 'learning' in adaptive layered networks and the fitting of data with high dimensional surfaces is discussed, leading naturally to a picture of 'generalization in terms of interpolation between known data points and suggests a rational approach to the theory of such networks.
IEEE Transactions on Information TheoryUniversal approximation bounds for superpositions of a sigmoidal function
2,849 Citations1993Andrew R. Barron
The approximation rate and the parsimony of the parameterization of the networks are shown to be advantageous in high-dimensional settings and the integrated squared approximation error cannot be made smaller than order 1/n/sup 2/d/ uniformly for functions satisfying the same smoothness assumption.
IEEE Transactions on Neural NetworksUniversal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes
2,630 Citations2006Guang-Bin Huang, Lei Chen +1 more
This paper proves in an incremental constructive method that in order to let SLFNs work as universal approximators, one may simply randomly choose hidden nodes and then only need to adjust the output weights linking the hidden layer and the output layer.
NeurocomputingConvex incremental extreme learning machine
1,136 Citations2007Guang-Bin Huang, Lihui Chen
This paper shows that while retaining the same simplicity, the convergence rate of I-ELM can be further improved by recalculating the output weights of the existing nodes based on a convex optimization method when a new hidden node is randomly added.
IEEE Transactions on Neural NetworksStochastic choice of basis functions in adaptive function approximation and the functional-link net
957 Citations1995B. Igelnik, Yoh‐Han Pao
A theoretical justification for the random vector version of the functional-link (RVFL) net is presented, based on a general approach to adaptive function approximation, which results are that the RVFL is a universal approximator for continuous functions on bounded finite dimensional sets.
IEEE Transactions on Neural NetworksCapabilities of a four-layered feedforward neural network: four layers versus three
416 Citations1997Shinichi Tamura, Machiko Tateishi
A proof is given showing that a three-layered feedforward network with N-1 hidden units can give any N input-target relations exactly, and a four-layering network is constructed and is found to give anyN input- target relations with a negligibly small error using only (N/2)+3 hidden units.
Journal of ComplexityOn the capabilities of multilayer perceptrons
380 Citations1988Eric B. Baum
A construction is presented here for implementing an arbitrary dichotomy with one hidden layer containing [ N d ] units, for any set of N points in general position in d dimensions, which is in fact the smallest such net as dichotomies which cannot be implemented by any net with fewer units.
NeurocomputingFully complex extreme learning machine
342 Citations2005Ming-Bin Li, Guang-Bin Huang +2 more
The simulation results show that the ELM equalizer significantly outperforms other neural network equalizers such as the complex minimal resource allocation network (CMRAN), complex radial basis function (CRBF) network and complex backpropagation (CBP) equalizers.
IEEE Transactions on Neural NetworksSmooth Function Approximation Using Neural Networks
318 Citations2005Silvio Ferrari, Robert F. Stengel
An algebraic approach for representing multidimensional nonlinear functions by feedforward neural networks is presented and shows that algebraic training is characterized by faster execution speeds and better generalization properties than contemporary optimization techniques.
Extreme learning machine: RBF network case
276 Citations2005Guang-Bin Huang, Chee‐Kheong Siew
This paper shows that ELM can be extended to radial basis function (RBF) network case, which allows the centers and impact widths of RBF kernels to be randomly generated and the output weights to be simply analytically calculated instead of iteratively tuned.
IEEE Transactions on Circuits and Systems II Analog and Digital Signal ProcessingCan threshold networks be trained directly?
267 Citations2006Guang-Bin Huang, Qinyu Zhu +4 more
It is shown theoretically that the recently developed extreme learning machine (ELM) algorithm can be used to train the neural networks with threshold functions directly instead of approximating them with sigmoid functions.
IEEE Transactions on Neural NetworksReal-Time Learning Capability of Neural Networks
255 Citations2006Guang-Bin Huang, Qinyu Zhu +1 more
A simple learning algorithm capable of real-time learning which can automatically select appropriate values of neural quantizers and analytically determine the parameters (weights and bias) of the network at one time only is proposed.
Neural ComputationApproximation by Fully Complex Multilayer Perceptrons
245 Citations2003Taehwan Kim, Tülay Adalı
Three proofs of the approximation capability of the fully complex MLP are provided based on the characteristics of singularity among ETFs, which shows the output of complex MLPs using ETFs with isolated and essential singularities uniformly converges to any nonlinear mapping in the deleted annulus of singularities nearest to the origin.
IEEE Journal on Selected Areas in CommunicationsChannel equalization using adaptive complex radial basis function networks
218 Citations1995Inhyok Cha, S.A. Kassam
Results of simulations show that the CRBFN with the stochastic-gradient algorithm can be quite effective in channel equalization.
Neural NetworksMultilayer Perceptrons to Approximate Quaternion Valued Functions
155 Citations1997P. Arena, Luigi Fortuna +2 more
A new density theorem is reported which makes HMLPs universal interpolators of quaternion valued continuous functions and the proof of the density theorem can be restricted in order to formulate a density theorem in the complex space.
Signal ProcessingComplex-valued radial basis function network, Part II: Application to digital communications channel equalisation
139 Citations1994Sheng Chen, Stephen McLaughlin +1 more
It is shown that the optimal Bayesian equaliser is structurally equivalent to the complex radial basis function network, and this intimate connection is exploited to develop fast training algorithms for implementing a Bayesianequaliser based on the latter.
IEEE Transactions on Neural NetworksCommunication channel equalization using complex-valued minimal radial basis function neural networks
132 Citations2002Deng Jianping, N. Sundararajan +1 more
A complex radial basis function neural network is proposed for equalization of quadrature amplitude modulation (QAM) signals in communication channels by utilizing a sequential learning algorithm and has the ability to grow and prune the (complex) RBF network's hidden neurons to ensure a parsimonious network structure.
Signal ProcessingComplex-valued radial basic function network, Part I: Network architecture and learning algorithms
132 Citations1994Sheng Chen, Stephen McLaughlin +1 more
A complex radial basis function network has complex centres and connection weights, but the nonlinearity of its hidden nodes remains a real-valued function, and it provides a powerful tool for nonlinear signal processing involving complex signals.
International Conference on Artificial Neural NetworksAdaptive radial basis function nonlinearities, and the problem of generalisation
123 Citations1989David Lowe
It is concluded that nonlinear optimisation of the first layer parameters is beneficial only when a minimal network is required to solve a given problem, since the same generalisation performance can be achieved simply by using more centres and adapting only the final layer by linear optimisation.
IEEE Transactions on Neural NetworksComments on "Stochastic choice of basis functions in adaptive function approximation and the functional-link net" [with reply]
62 Citations1997Jinyan Li, W Chow +2 more
This paper includes some comments and amendments of the above-mentioned paper by Igelnik et al. (1995), and acknowledges that a factor of two was omitted in the statement of a trigonometric identity.
Fully complex backpropagation for constant envelope signal processing
37 Citations2002Tae-Hwan Kim, Tülay Adalı
A feedforward neural network architecture employing hyperbolic tangent tanh(z) function defined in the entire complex domain, which can easily outperform the non-analytic split complex activation function in convergence speed and achievable minimum squared error when the domain is bounded around the unit circle.
International Journal of Neural SystemsMULTILAYER PERCEPTRONS TO APPROXIMATE COMPLEX VALUED FUNCTIONS
33 Citations1995Paolo Arena, Luigi Fortuna +2 more
A new density theorem for Complex Multilayer Perceptrons with complex valued non-analytical sigmoidal activation functions has been proven, which makes Multilayered neurons universal interpolators of continuous complex valued functions.
IEEE International Conference on Acoustics Speech and Signal ProcessingUniversal approximation of fully complex feed-forward neural networks
25 Citations2002Taehwan Kim, Tülay Adalı
It is shown that folly complex FNNs can universally approximate any complex mapping to an arbitrary accuracy on a compact set of input patterns with probability 1.