Diversity measures for multiple classifier system analysis and design
Information FusionPublished 29 May 2004
Terry Windeatt
Citations106
SJR quartileQ1
SJR score4.13
SNIP4.60
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A new pair-wise measure, which is computed between pairs of patterns rather than pairs of classifiers, is also proposed for two-class problems, and it is shown experimentally that the proposed measure is well correlated with base classifier test error as base classifiers complexity is systematically varied.
Abstract
In the context of Multiple Classifier Systems, diversity among base classifiers is known to be a necessary condition for improvement in ensemble performance. In this paper the ability of several pair-wise diversity measures to predict generalisation error is compared. A new pair-wise measure, which is computed between pairs of patterns rather than pairs of classifiers, is also proposed for two-class problems. It is shown experimentally that the proposed measure is well correlated with base classifier test error as base classifier complexity is systematically varied. However, correlation with unity-weighted sum and vote is shown to be weaker, demonstrating the difficulty in choosing base classifier complexity for optimal fusion. An alternative strategy based on weighted combination is also investigated and shown to be less sensitive to number of training epochs.
Keywords
Computer Science
IEEE Transactions on Pattern Analysis and Machine IntelligenceNeural network ensembles
4,269 Citations1990Lars Kai Hansen, Peter Salamon
It is shown that the remaining residual generalization error can be reduced by invoking ensembles of similar networks, which helps improve the performance and training of neural networks for classification.
The Annals of StatisticsBoosting the margin: a new explanation for the effectiveness of voting methods
2,313 Citations1998Peter L. Bartlett, Yoav Freund +2 more
It is shown that techniques used in the analysis of Vapnik's support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error.
Machine LearningMeasures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy
2,288 Citations2003Ludmila I. Kuncheva, Christopher J. Whitaker
Although there are proven connections between diversity and accuracy in some special cases, the results raise some doubts about the usefulness of diversity measures in building classifier ensembles in real-life pattern recognition problems.
The Annals of StatisticsArcing classifier (with discussion and a rejoinder by the author)
1,105 Citations1998Leo Breiman
Two arcing algorithms are explored, compared to each other and to bagging, and the definitions of bias and variance for a classifier as components of the test set error are introduced.
IEEE Transactions on Pattern Analysis and Machine IntelligenceA theoretical study on six classifier fusion strategies
713 Citations2002Ludmila I. Kuncheva
Formulas for the classification error for the following fusion methods: average, minimum, maximum, median, majority vote, and oracle are given.
The Annals of StatisticsEmpirical Margin Distributions and Bounding the Generalization Error of Combined Classifiers
468 Citations2002Vladimir Koltchinskii, Dmitry Panchenko
New probabilistic upper bounds on generalization error of complex classifiers that are combinations of simple classifier combinations, based on the methods of the theory of Gaussian and empirical processes are proved.
Elsevier eBooksError-Correcting Output Coding Corrects Bias and Variance
357 Citations1995Eun Bae Kong, Thomas G. Dietterich
An investigation of why the ECOC technique works, particularly when employed with decision-tree learning algorithms, shows that it can reduce the variance of the learning algorithm.
IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics)Switching between selection and fusion in combining classifiers: an experiment
351 Citations2002Ludmila I. Kuncheva
A combination of classifier selection and fusion by using statistical inference to switch between the two by offering a discussion on when to combine classifiers and how classifiers selection (static or dynamic) can be misled by the differences in the classifier team.
Machine LearningUsing Correspondence Analysis to Combine Classifiers
210 Citations1999Christopher J. Merz
The method described uses the strategies of stacking and Correspondence Analysis to model the relationship between the learning examples and their classification by a collection of learned models, and a nearest neighbor method is applied within the resulting representation to classify previously unseen examples.
IEEE Transactions on Pattern Analysis and Machine IntelligenceOptimal linear combination of neural networks for improving classification performance
205 Citations2000N. Ueda
Experimental results show that the proposed method can construct a better combined classifier that outperforms the best single classifier in terms of overall classification errors for test data.
Machine LearningVariance and Bias for General Loss Functions
162 Citations2003Gareth James
This paper suggests an explicit list of rules that any “reasonable” set of definitions should satisfy and produces bias and variance definitions which generalize to any symmetric loss function.
Information FusionAn experimental study on diversity for bagging and boosting with linear classifiers
161 Citations2002Ludmila I. Kuncheva, Marina Skurichina +1 more
Diversity measures indicated that Boosting succeeds in inducing diversity even for stable classifiers whereas Bagging does not, confirming in a quantitative way the intuitive explanation behind the success of Boosting for linear classifiers for increasing training sizes, and the poor performance of Bagging.
Information FusionCoding and decoding strategies for multi-class learning problems
130 Citations2003Terry Windeatt, Reza Ghaderi
The binary (0,1) code matrix conditions necessary for reduction of error in the ECOC framework are considered, and it is shown that equidistant codes can be generated by using properties related to the number of 1s in each row and between any pair of rows.
Lecture notes in computer scienceThat Elusive Diversity in Classifier Ensembles
127 Citations2003Ludmila I. Kuncheva
It is argued that even without a clear-cut definition and theory behind it, studying diversity may prompt viable heuristic solutions in classifier ensembles.
Lecture notes in computer scienceData Dependence in Combining Classifiers
57 Citations2003Mohamed S. Kamel, Nayer Wanas
A new categorization of combining schemes based on their dependence on the data patterns being classified is presented, arguing that data dependent, and especially explicitly datadependent, approaches represent the highest potential for improved performance.
Pattern RecognitionVote counting measures for ensemble classifiers
35 Citations2003Terry Windeatt
A different measure based on a spectral representation is proposed for two-class problems, which incorporates terms representing positive and negative correlation of pairs of training patterns with respect to class labels.
Lecture notes in computer scienceLinear Combiners for Classifier Fusion: Some Theoretical and Experimental Results
26 Citations2003Giorgio Fumera, Fabio Roli
The theoretical and experimental analysis of two widely used combining rules, namely, the simple and weighted average of classifier outputs, show that they can be useful in real applications, for designing multiple classifier systems based on linear combiners.
Pattern Recognition LettersSpectral technique for hidden layer neural network training
13 Citations1997Terry Windeatt, Robert Tebbs
Weight constraints derived from a spectral summation are used to check separability during the partitioning phase, and to limit hyperplane movement during training.
Neural Processing LettersRecursive Partitioning Technique for Combining Multiple Classifiers
8 Citations2001Terry Windeatt
A recursive partitioning technique for analysing feature space of multiple classifier decisions and leaving out random subsets of ICS patterns from individual (base) classifier training sets is shown to improve performance of the combined classifiers.
Lecture notes in computer scienceSpectral Coefficients and Classifier Correlation
3 Citations2003Terry Windeatt, Reza Ghaderi +1 more
A measure based on counting votes to estimate first order spectral coefficients for two-class problems is described and used to design a weighted vote combiner, shown experimentally to be less sensitive than majority vote to base classifier complexity.