ModEnPBT: A Modified Backtracking Ensemble Pruning algorithm
Applied Soft ComputingPublished 4 July 2013
Qun Dai, Zhuan Liu
Citations21
SJR quartileQ1
SJR score1.51
SNIP1.97
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Experimental results on three benchmark classification tasks demonstrate the validity and effectiveness of the proposed ModEnPBT, which is based upon the design idea of Ensemble Pruning via Backtracking algorithm, and however, aiming at overcoming its drawback of redundant solution space definition.
Abstract
This paper proposes a new Modified Backtracking Ensemble Pruning algorithm (ModEnPBT), which is based upon the design idea of our previously proposed Ensemble Pruning via Backtracking algorithm (EnPBT), and however, aiming at overcoming its drawback of redundant solution space definition. Solution space of ModEnPBT is compact with no repeated solution vectors, therefore it possesses relatively higher searching efficiency compared with EnPBT algorithm. ModEnPBT still belongs to the category of Backtracking algorithm, which can systematically search for the solutions of a problem in a manner of depth-first, which is suitable for solving all those large-scale combinatorial optimization problems. Experimental results on three benchmark classification tasks demonstrate the validity and effectiveness of the proposed ModEnPBT.
Keywords
Computer Science
Proceedings of the 25th international conference on Machine learning
5,542 Citations2008William W. Cohen, Andrew McCallum +1 more
Artificial IntelligenceEnsembling neural networks: Many could be better than all
1,982 Citations2002Zhi‐Hua Zhou, Jianxin Wu +1 more
The bias-variance decomposition of the error is provided in this paper, which shows that the success of GASEN may lie in that it can significantly reduce the bias as well as the variance.
Ensemble selection from libraries of models
844 Citations2004Rich Caruana, Alexandru Niculescu-Mizil +2 more
A method for constructing ensembles from libraries of thousands of models using forward stepwise selection to be optimized to performance metric such as accuracy, cross entropy, mean precision, or ROC Area is presented.
Pruning Adaptive Boosting
512 Citations1997Dragos D. Margineantu, Thomas G. Dietterich
IEEE Transactions on Pattern Analysis and Machine IntelligenceAn Analysis of Ensemble Pruning Techniques Based on Ordered Aggregation
317 Citations2009Gonzalo Martínez-Muñoz, Daniel Hernández-Lobato +1 more
The results of this empirical investigation show that ordered aggregation can be used for the efficient generation of pruned ensembles that are competitive, in terms of performance and robustness of classification, with computationally more costly methods that directly select optimal or near-optimal subensembles.
Information FusionEnsemble diversity measures and their application to thinning
216 Citations2004Robert E. Banfield, Lawrence Hall +2 more
The percentage correct diversity measure shows a greatest correlation between the increase in voted ensemble accuracy and the diversity value, and the methods proposed for thinning show that ensembles can be made smaller without loss in accuracy.
Pruning in ordered bagging ensembles
178 Citations2006Gonzalo Martínez-Muñoz, Alberto Suárez
Puned subensembles containing between 15% and 30% of the initial pool of classifiers, besides being smaller, improve the generalization performance of the full bagging ensemble in the classification problems investigated.
Lecture notes in computer scienceDiversity Regularized Ensemble Pruning
139 Citations2012Nan Li, Yang Yu +1 more
A theoretical study on the effect of diversity on the generalization performance of voting in the PAC-learning framework and applies explicit diversity regularization to ensemble pruning, and proposes the Diversity Regularized Ensemble Pruning (DREP) method.
Lecture notes in computer scienceSelective Ensemble of Decision Trees
117 Citations2007Zhi‐Hua Zhou, Wei Tang
Experiments show that ensembles generated by a selective ensemble algorithm, which selects some of the trained C4.5 decision trees to make up an ensemble, may be not only smaller in the size but also stronger in the generalization than ensemble generated by non-selective algorithms.
Pattern Recognition LettersUsing boosting to prune bagging ensembles
110 Citations2006Gonzalo Martínez-Muñoz, Alberto Suárez
In all the classification problems investigated pruned ensembles with 20% of the original classifiers show statistically significant improvements over bagging, and, if no information about the level of noise is available, it is a robust alternative to AdaBoost.
Machine LearningAn ensemble uncertainty aware measure for directed hill climbing ensemble pruning
101 Citations2010Ioannis Partalas, Grigorios Tsoumakas +1 more
Empirical results show that using the proposed measure to prune a heterogeneous ensemble leads to significantly better accuracy results compared to state-of-the-art measures and other baseline methods, while keeping only a small fraction of the original models.
IEEE Transactions on Neural NetworksUniformly Stable Backpropagation Algorithm to Train a Feedforward Neural Network
93 Citations2011José de Jesús Rubio, Plamen Angelov +1 more
A theorem is stated and proven which guarantees uniform stability of a general discrete-time system and the backpropagation (BP) algorithm with a new time-varying rate is uniformly stable for online identification and the identification error converges to a small zone bounded by the uncertainty.
Lecture notes in computer scienceOn the Boosting Pruning Problem
92 Citations2000Christino Tamon, Jie Xiang
It is pointed out that the boosting pruning problem is intractable even to approximate and a margin-based theoretical heuristic is suggested for this problem.
IET Control Theory and ApplicationsModified optimal control with a backpropagation network for robotic arms
83 Citations2012José de Jesús Rubio
In this study, the trajectory tracking problem of robotic arms is considered and two novel modified optimal controllers based on neural networks are proposed by means of a Lyapunov-like analysis.
NeurocomputingPruning an ensemble of classifiers via reinforcement learning
81 Citations2008Ioannis Partalas, Grigorios Tsoumakas +1 more
A new pruning approach that uses the Q-learning algorithm in order to approximate an optimal policy of choosing whether to include or exclude each classifier from the ensemble is contributed.
Knowledge-Based SystemsA competitive ensemble pruning approach based on cross-validation technique
77 Citations2012Qun Dai
This paper proposes a new Competitive technique for Ensemble Pruning based on Cross-Validation (CEPCV), which might alleviate the local minima problem of many conventional ensemble pruning approaches only at the cost of a little greater computational cost, which is acceptable to most applications of ensemble learning.
Knowledge and Information SystemsCost Complexity-Based Pruning of Ensemble Classifiers
72 Citations2001Andreas L. Prodromidis, Salvatore J. Stolfo
This paper describes an algorithm for pruning the ensemble meta-classifier as a means to reduce its size while preserving its accuracy and presents a technique for measuring the trade-off between predictive performance and available run-time system resources.
NeurocomputingBoundedness and convergence of batch back-propagation algorithm with penalty for feedforward neural networks
60 Citations2012Huisheng Zhang, Wei Wu +1 more
The main contribution of this paper is to theoretically prove the boundedness of the weights in the network training process, which is used to prove some convergence results of the algorithm, which cover both the weak and strong convergence.
NeurocomputingEmpirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles
44 Citations2011Daniel Hernández-Lobato, Gonzalo Martínez-Muñoz +1 more
This article analyzes two approximate techniques especially devised to address the problem of identifying the optimal subset of regressors in a regression bagging ensemble and constructs a relaxed version of the problem that can be solved using semidefinite programming.
IEEE Transactions on Neural NetworksIdentification of Extended Hammerstein Systems Using Dynamic Self-Optimizing Neural Networks
43 Citations2011Xuemei Ren, Xiaohua Lv
A new dynamic self-optimizing neural network with online adjusting hidden layer and weights is proposed for a class of extended Hammerstein systems with non-Gaussian noises and can be established without a priori knowledge of the unknown nonlinearity.
NeurocomputingA novel ensemble pruning algorithm based on randomized greedy selective strategy and ballot
37 Citations2013Qun Dai
Experimental results, including t-tests on the three benchmark classification tasks, verified the validity of the proposed RGSS&B-EP algorithm and introduced randomization technique into the procedure of greedy ensemble pruning.
Neural Computing and ApplicationsA method for online pattern recognition of abnormal eye movements
26 Citations2011José de Jesús Rubio, Floriberto Ortiz-Rodríguez +2 more
A method for the pattern recognition is used to provide a better diagnosis for patients related with the abnormal eye movements and comparison results of the proposed method using each of the intelligent algorithms for online pattern recognition of two eye movements are presented.
Lecture notes in computer scienceInterpretable neural networks with BP-SOM
24 Citations1998Ton Weijters, Antal van den Bosch +1 more
It is shown that networks trained with BP-SOM show interesting regularities, in that hidden-unit activations become restricted to discrete values, and that the som part can be exploited for automatic rule extraction.
Neural Processing LettersThe BP-SOM architecture and learning rule
19 Citations1995A.J.M.M. Weijters
Results on two hard to learn classification tasks show that the BP-SOM architecture and learning rule offer a strong alternative for training multilayer feedforward networks with back-propagation.
Neural Processing LettersImproved CBP Neural Network Model with Applications in Time Series Prediction
17 Citations2003Dai Qun, Chen Songcan +1 more
A more generalized network model ICBP (Improved Circular Back-Propagation Neural Network) was designed by adding an extensive node with quadratic form to the original CBP inputs and endowing fixed values to the weights between this node and all the hidden nodes.
NeurocomputingThe build of n-Bits Binary Coding ICBP Ensemble System
11 Citations2011Qun Dai, Ningzhong Liu
The proposed nBBC-ICBP-ES is simple but efficient and effective, and also potentially significant for the applications of NN ensemble, and assembling many small NNs might be better than just utilizing an individual large one, and further, assembling many heterogeneous small NNS might bebetter than assembling many homogeneous ones.