High-Performance Extreme Learning Machines: A Complete Toolbox for Big Data Applications
IEEE AccessPublished 1 January 2015Open access
Anton Akusok, Kaj-Mikael Björk, Yoan Miché, Amaury Lendasse
Citations266
SJR quartileQ1
SJR score0.85
SNIP1.50
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper presents a complete approach to a successful utilization of a high-performance extreme learning machines (ELM) Toolbox for Big Data, and summarizes recent advantages in algorithmic performance; gives a fresh view on the ELM solution in relation to the traditional linear algebraic performance; and reaps the latest software and hardware performance achievements.
Abstract
This paper presents a complete approach to a successful utilization of a high-performance extreme learning machines (ELMs) Toolbox for Big Data. It summarizes recent advantages in algorithmic performance; gives a fresh view on the ELM solution in relation to the traditional linear algebraic performance; and reaps the latest software and hardware performance achievements. The results are applicable to a wide range of machine learning problems and thus provide a solid ground for tackling numerous Big Data challenges. The included toolbox is targeted at enabling the full potential of ELMs to the widest range of users.
Keywords
Computer ScienceEngineering
ACM Transactions on Intelligent Systems and TechnologyLIBSVM
41,340 Citations2011Chih-Chung Chang, Chih‐Jen Lin
Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.
Machine LearningSupport-vector networks
40,463 Citations1995Corinna Cortes, Vladimir Vapnik
High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
Machine LearningSupport-Vector Networks
33,035 Citations1995Corinna Cortes, Vladimir Vapnik
High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
Neural Networks: A Comprehensive Foundation
29,816 Citations1998Simon Haykin
Thorough, well-organized, and completely up to date, this book examines all the important aspects of this emerging technology, including the learning process, back-propagation learning, radial-basis function networks, self-organizing systems, modular networks, temporal processing and neurodynamics, and VLSI implementation of neural networks.
Journal of Electronic ImagingPattern Recognition and Machine Learning
21,976 Citations2007Christopher Bishop
Probability Distributions, linear models for Regression, Linear Models for Classification, Neural Networks, Graphical Models, Mixture Models and EM, Sampling Methods, Continuous Latent Variables, Sequential Data are studied.
NeurocomputingExtreme learning machine: Theory and applications
13,180 Citations2006Guang-Bin Huang, Qinyu Zhu +1 more
A new learning algorithm called ELM is proposed for feedforward neural networks (SLFNs) which randomly chooses hidden nodes and analytically determines the output weights of SLFNs which tends to provide good generalization performance at extremely fast learning speed.
Psychological ReviewThe perceptron: A probabilistic model for information storage and organization in the brain.
11,746 Citations1958Frank F. Rosenblatt
This article will be concerned primarily with the second and third questions, which are still subject to a vast amount of speculation, and where the few relevant facts currently supplied by neurophysiology have not yet been integrated into an acceptable theory.
The Annals of StatisticsLeast angle regression
9,493 Citations2004Bradley Efron, Trevor Hastie +2 more
TechnometricsRidge Regression: Biased Estimation for Nonorthogonal Problems
6,562 Citations2000Arthur E. Hoerl, Robert W. Kennard
IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics)Extreme Learning Machine for Regression and Multiclass Classification
5,493 Citations2011Guang-Bin Huang, Hongming Zhou +2 more
ELM provides a unified learning platform with a widespread type of feature mappings and can be applied in regression and multiclass classification applications directly and in theory, ELM can approximate any target continuous function and classify any disjoint regions.
Lecture notes in computer scienceGaussian Processes in Machine Learning
5,023 Citations2004Carl Edward Rasmussen
A basic introduction to Gaussian Process regression models is given and the simple equations for incorporating training data and how to learn the hyperparameters using the marginal likelihood are examined.
Extreme learning machine: a new learning scheme of feedforward neural networks
4,112 Citations2005Guang-Bin Huang, Qin‐Yu Zhu +1 more
A new learning algorithm called extreme learning machine (ELM) for single-hidden layer feedforward neural networks (SLFNs) which randomly chooses the input weights and analytically determines the output weights of SLFNs is proposed.
IEEE Transactions on Neural NetworksOrthogonal least squares learning algorithm for radial basis function networks
3,361 Citations1991Sheng Chen, C.F.N. Cowan +1 more
The authors propose an alternative learning procedure based on the orthogonal least-squares method, which provides a simple and efficient means for fitting radial basis function networks.
IEEE Transactions on Neural NetworksUniversal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes
2,630 Citations2006Guang-Bin Huang, Lei Chen +1 more
This paper proves in an incremental constructive method that in order to let SLFNs work as universal approximators, one may simply randomly choose hidden nodes and then only need to adjust the output weights linking the hidden layer and the output layer.
TechnometricsRidge Regression: Applications to Nonorthogonal Problems
2,624 Citations1970Arthur E. Hoerl, Robert W. Kennard
International Journal of Data Warehousing and MiningMulti-Label Classification
2,484 Citations2007Grigorios Tsoumakas, Ioannis Katakis
The task of multi-label classification is introduced, the sparse related literature is organizes into a structured presentation and comparative experimental results of certain multilabel classification methods are performed.
IEEE Transactions on Neural NetworksA Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks
1,974 Citations2006Nanying Liang, Guang-Bin Huang +2 more
The results show that the OS-ELM is faster than the other sequential algorithms and produces better generalization performance on benchmark problems drawn from the regression, classification and time series prediction areas.
NeurocomputingLearning and generalization characteristics of the random vector functional-link net
1,128 Citations1994Yoh‐Han Pao, Gwang-Hoon Park +1 more
The learning and generalization characteristics of the random vector version of the Functional-link net are explored and compared with those attainable with the GDR algorithm and it seems that ‘ overtraining ’ occurs for stochastic mappings.
IEEE Transactions on Neural NetworksStochastic choice of basis functions in adaptive function approximation and the functional-link net
957 Citations1995B. Igelnik, Yoh‐Han Pao
A theoretical justification for the random vector version of the functional-link (RVFL) net is presented, based on a general approach to adaptive function approximation, which results are that the RVFL is a universal approximator for continuous functions on bounded finite dimensional sets.
Cognitive ComputationAn Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels
957 Citations2014Guang-Bin Huang
An insight into ELMs in three aspects, viz: random neurons, random features and kernels is provided and it is shown that in theory ELMs (with the same kernels) tend to outperform support vector machine and its variants in both regression and classification applications with much easier implementation.
IEEE Transactions on Neural NetworksOP-ELM: Optimally Pruned Extreme Learning Machine
759 Citations2009Yoan Miché, A. Sorjamaa +4 more
The proposed OP-ELM methodology performs several orders of magnitude faster than the other algorithms used in this brief, except the original ELM, and is still able to maintain an accuracy that is comparable to the performance of the SVM.
IEEE Transactions on Pattern Analysis and Machine IntelligenceSkin segmentation using color pixel classification: analysis and comparison
758 Citations2004Son Lam Phung, Abdesselam Bouzerdoum +1 more
This work finds that color quantization can be as low as 64 bins per channel, although higher histogram sizes give better segmentation performance, and that the Bayesian classifier with the histogram technique and the multilayer perceptron classifier are found to perform better compared to other tested classifiers.
NeurocomputingWeighted extreme learning machine for imbalance learning
740 Citations2012Weiwei Zong, Guang-Bin Huang +1 more
A weighted ELM which is able to deal with data with imbalanced class distribution while maintain the good performance on well balanced data as unweighted ELM and generalized to cost sensitive learning.
IEEE Intelligent SystemsExtreme Learning Machines [Trends & Controversies]
702 Citations2013Erik Cambria, Guang-Bin Huang +30 more
Extreme Learning Machine
626 Citations2013Erik Cambria, Guang-Bin Huang +30 more
Feedforward neural networks with random weights
544 Citations2003Walter Schmidt, Martin A. Kraaijveld +1 more
It is shown that a large fraction of the parameters (the weights of neural networks) are of less importance and do not need to be measured with high accuracy and therefore the reported experiments seem to be more realistic from a classical point of view.
Cognitive ComputationWhat are Extreme Learning Machines? Filling the Gap Between Frank Rosenblatt’s Dream and John von Neumann’s Puzzle
474 Citations2015Guang-Bin Huang
ELM theories manage to address the open problem which has puzzled the neural networks, machine learning and neuroscience communities for 60 years: whether hidden nodes/neurons need to be tuned in learning, and proved that in contrast to the common knowledge and conventional neural network learning tenets,hidden nodes/NEurons do not need to been iteratively tuned in wide types of neural networks and learning models.
IEEE Computational Intelligence MagazineLocal Receptive Fields Based Extreme Learning Machine
347 Citations2015Guang-Bin Huang, Zuo Bai +2 more
The general architecture of locally connected ELM is studied, showing that: 1) ELM theories are naturally valid for local connections, thus introducing local receptive fields to the input layer; 2) each hidden node in ELM can be a combination of several hidden nodes (a subnetwork), which is also consistent with ELM theory.
IEEE Transactions on Circuits and Systems II Analog and Digital Signal ProcessingCan threshold networks be trained directly?
267 Citations2006Guang-Bin Huang, Qinyu Zhu +4 more
It is shown theoretically that the recently developed extreme learning machine (ELM) algorithm can be used to train the neural networks with threshold functions directly instead of approximating them with sigmoid functions.
IEEE Computational Intelligence MagazineThe Emerging "Big Dimensionality"
248 Citations2014Yiteng Zhai, Yew-Soon Ong +1 more
The state-of-the-art feature selection schemes reported in the field of computational intelligence are reviewed to reveal the inadequacies of existing approaches in keeping pace with the emerging phenomenon of Big Dimensionality.
NeurocomputingTROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization
225 Citations2011Yoan Miché, Mark van Heeswijk +3 more
The proposed modification of the OP-ELM uses a cascade of two regularization penalties: first a L1 penalty to rank the neurons of the hidden layer, followed by a L2 penalty on the regression weights for numerical stability and efficient pruning of the neurons.
An additional hidden unit test for neglected nonlinearity in multilayer feedforward networks
210 Citations1989White
The author presents a statistical test of the hypothesis that a given multilayer feedforward network exactly represents some unknown mapping subject to inherent noise against the alternative that the network neglects some nonlinear structure in the mapping, leading to potentially avoidable approximation errors.
NeurocomputingGPU-accelerated and parallelized ELM ensembles for large-scale regression
163 Citations2011Mark van Heeswijk, Yoan Miché +2 more
The proposed approach for performing regression on large data sets in reasonable time, using an ensemble of extreme learning machines (ELMs) is not limited to a specific type of ELM and can be employed for a large variety of ELMs.
NeurocomputingRegularized extreme learning machine for regression with missing data
118 Citations2012Qi Yu, Yoan Miché +4 more
A cascade of L"1 penalty (LARS) and L"2 penalty (Tikhonov regularization) on ELM (TROP-ELM) to regularize the matrix computations and hence makes the MSE computation more reliable, and estimates the expected pairwise distances between samples directly on incomplete data so that it offers the ELM a solution to solve the missing data issues.
Lecture notes in computer scienceMultiresponse Sparse Regression with Application to Multidimensional Scaling
113 Citations2005Timo Similä, Jarkko Tikka
This work considers a general setting in which both the target and regressors may be multivariate, and chooses sequentially those regressors that allow, on average, the best predictions of all the target variables.
Lecture notes in computer scienceAdaptive Ensemble Models of Extreme Learning Machines for Time Series Prediction
107 Citations2009Mark van Heeswijk, Yoan Miché +5 more
In the experiments, it is shown that the adaptive ensemble model achieves a test error comparable to the best methods, while keeping adaptivity, and has low computational cost.
A methodology for Building Regression Models using Extreme Learning Machine: OP-ELM
66 Citations2008Yoan Miché, Patrick Bas +3 more
OP-ELM is proposed: the network is first created using Extreme Learning Process, selection of the most relevant nodes is performed using Least Angle Regression (LARS) ranking of the nodes and a Leave-One-Out estimation of the performances.
NeurocomputingOnline and adaptive pseudoinverse solutions for ELM weights
50 Citations2014André van Schaik, Jonathan Tapson
This paper discusses incremental methods for solving the pseudoinverse which are suitable for ELM and shows that careful choice of methods allows us to optimize for accuracy, ease of computation, or adaptability of the solution.
Cognitive ComputationA Two-Stage Methodology Using K-NN and False-Positive Minimizing ELM for Nominal Data Classification
36 Citations2014Anton Akusok, Yoan Miché +3 more
Experimental results using a specific dataset provided by F-Secure Corporation show that this methodology provides a rapid decision on new samples, with a direct control over the false positives and thus on the decision capabilities of the model.
IEEE Computational Intelligence MagazineArbitrary Category Classification of Websites Based on Image Content
27 Citations2015Anton Akusok, Yoan Miché +4 more
The methodology is robust to noise and can learn abstract target categories; website classification accuracy surpasses 97% for the most important categories considered in this study.
The 2006 IEEE International Joint Conference on Neural Network ProceedingsCommon Subset Selection of Inputs in Multiresponse Regression
17 Citations2006Timo Similä, Jarkko Tikka
The multiresponse sparse regression algorithm is an input selection method for linearly parameterized models, which updates with carefully chosen step lengths, and competes favorably with other methods when many correlated inputs are available for model construction.