Handling imbalanced datasets: A review
Published 1 January 2006
Sotiris Kotsiantis, Dimitris Kanellopoulos, Panayiotis Pintelas
Citations880
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper describes various techniques for handling imbalanced dataset problems, and hopes that the references cited will cover the major theoretical issues, guiding the researcher in interesting re- search directions and suggesting possible bias combinations that have yet to be explored.
Abstract
Abstract. Learning classifiers from imbalanced or skewed datasets is an important topic, arising very often in practice in classification problems. In such problems, almost all the instances are labelled as one class, while far fewer instances are labelled as the other class, usually the more important class. It is obvious that traditional classifiers seeking an accurate performance over a full range of instances are not suitable to deal with imbalanced learning tasks, since they tend to classify all the data into the majority class, which is usually the less important class. This paper describes various techniques for handling imbalance dataset problems. Of course, a single article cannot be a complete review of all the methods and algorithms, yet we hope that the references cited will cover the major theoretical issues, guiding the researcher in interesting research directions and suggesting possible bias combinations that have yet to be explored. 1
Keywords
Computer ScienceHealth ProfessionsEngineering
Journal of Artificial Intelligence ResearchSMOTE: Synthetic Minority Over-sampling Technique
31,194 Citations2002Nitesh V. Chawla, Kevin W. Bowyer +2 more
A combination of the method of oversampling the minority (abnormal) class and under-sampling the majority class can achieve better classifier performance (in ROC space) and a combination of these methods and the area under the Receiver Operating Characteristic curve (AUC) and the ROC convex hull strategy is evaluated.
Pattern RecognitionThe use of the area under the ROC curve in the evaluation of machine learning algorithms
7,233 Citations1997Andrew P. Bradley
AUC exhibits a number of desirable properties when compared to overall accuracy: increased sensitivity in Analysis of Variance (ANOVA) tests; a standard error that decreased as both AUC and the number of test samples increased; decision threshold independent; and it is invariant to a priori class probabilities.
ACM SIGKDD Explorations NewsletterA study of the behavior of several methods for balancing machine learning training data
4,130 Citations2004Gustavo E. A. P. A. Batista, Ronaldo C. Prati +1 more
This work performs a broad experimental evaluation involving ten methods, three of them proposed by the authors, to deal with the class imbalance problem in thirteen UCI data sets, and shows that, in general, over-sampling methods provide more accurate results than under-sampled methods considering the area under the ROC curve (AUC).
Elsevier eBooksFast Effective Rule Induction
3,767 Citations1995William W. Cohen
This paper evaluates the recently-proposed rule learning algorithm IREP on a large and diverse collection of benchmark problems, and proposes a number of modifications resulting in an algorithm RIPPERk that is very competitive with C4.5 and C 4.5rules with respect to error rates, but much more efficient on large samples.
Intelligent Data AnalysisThe class imbalance problem: A systematic study1
3,298 Citations2002Nathalie Japkowicz, Shaju Stephen
The assumption that the class imbalance problem does not only affect decision tree systems but also affects other classification systems such as Neural Networks and Support Vector Machines is investigated.
Addressing the Curse of Imbalanced Training Sets: One-Sided Selection.
2,171 Citations1997Miroslav Kubát, Stan Matwin
Lecture notes in computer scienceSMOTEBoost: Improving Prediction of the Minority Class in Boosting
1,633 Citations2003Nitesh V. Chawla, Aleksandar Lazarević +2 more
This paper presents a novel approach for learning from imbalanced data sets, based on a combination of the SMOTE algorithm and the boosting procedure, which shows improvement in prediction performance on the minority class and overall improved F-values.
ACM SIGKDD Explorations NewsletterMining with rarity
1,386 Citations2004Gary M. Weiss
It is demonstrated that rare classes and rare cases are very similar phenomena---both forms of rarity are shown to cause similar problems during data mining and benefit from the same remediation methods.
MetaCost
1,294 Citations1999Pedro Domingos
A principled method for making an arbitrary classifier cost-sensitive by wrapping a cost-minimizing procedure around it is proposed, called MetaCost, which treats the underlying classifier as a black box, requiring no knowledge of its functioning or change to it.
Machine LearningRobust Classification for Imprecise Environments
1,206 Citations2001Foster Provost, Tom Fawcett
It is shown that it is possible to build a hybrid classifier that will perform at least as well as the best available classifier for any target conditions, and in some cases, the performance of the hybrid actually can surpass that of the best known classifier.
Computational IntelligenceA Multiple Resampling Method for Learning from Imbalanced Data Sets
1,022 Citations2004Andrew Estabrooks, Taeho Jo +1 more
It is concluded that combining different expressions of the resampling approach is an effective solution to the tuning problem and the proposed combination scheme is evaluated on imbalanced subsets of the Reuters‐21578 text collection and is shown to be quite effective for these problems.
Journal of Artificial Intelligence ResearchLearning When Training Data are Costly: The Effect of Class Distribution on Tree Induction
928 Citations2003Gary M. Weiss, Foster Provost
A "budget-sensitive" progressive sampling algorithm is introduced for selecting training examples based on the class associated with each example and it is shown that the class distribution of the resulting training set yields classifiers with good (nearly-optimal) classification performance.
C4.5, Class Imbalance, and Cost Sensitivity: Why Under-Sampling beats Over-Sampling
833 Citations2003Chris Drummond, Robert C. Holte
This paper shows that using C4.5 with undersampling establishes a reasonable standard for algorithmic comparison, and it is recommended that the cheapest class classifier be part of that standard as it can be better than under-sampling for relatively modest costs.
Controlling the Sensitivity of Support Vector Machines
691 Citations1999K Veropoulos, I C G Campbell +1 more
Two schemes for adjusting the sensitivity and speciicity of Support Vector Machines and the description of their performance using receiver operating characteristic (ROC) curves are discussed and their use on real-life medical diagnostic tasks is illustrated.
ACM SIGKDD Explorations NewsletterClass imbalances versus small disjuncts
681 Citations2004Taeho Jo, Nathalie Japkowicz
It is argued that, in order to improve classifier performance, it may be more useful to focus on the small disjuncts problem than it is tofocus on the class imbalance problem, and experiments suggest that the problem is not directly caused by class imbalances, but rather, that class imbalance may yield small disJuncts which will cause degradation.
ACM SIGKDD Explorations NewsletterLearning from imbalanced data sets with boosting and data generation
606 Citations2004Hongyu Guo, Herna L. Viktor
The approach does not sacrifice one class in favor of the other, but produces high predictions against both minority and majority classes, and compares well in comparison with a base classifier, a standard benchmarking boosting algorithm and three advanced boosting-based algorithms for imbalanced data set.
AdaCost: Misclassification Cost-Sensitive Boosting
596 Citations1999Wei Fan, Salvatore J. Stolfo +2 more
It is formally show that AdaCost reduces the upper bound of cumulative misclassification cost of the training set of the training set without consuming additional computing power.
Pattern RecognitionStrategies for learning in class imbalance problems
571 Citations2003R. Barandela, J. Salvador Sánchez +2 more
A set of examples or training set (TS) is said to be imbalanced if one of the classes is represented by a very small number of cases compared to the other classes.
ACM SIGKDD Explorations NewsletterFeature selection for text categorization on imbalanced data
550 Citations2004Zhaohui Zheng, Wu Xiaoyun +1 more
This work investigates the usefulness of explicit control of that combination within a proposed feature selection framework and shows both great potential and actual merits of explicitly combining positive and negative features in a nearly optimal fashion according to the imbalanced data.
Learning and making decisions when costs and probabilities are both unknown
455 Citations2001Bianca Zadrozny, Charles Elkan
This work explains how to obtain unbiased estimators for example-dependent costs, taking into account the difficulty that in general, probabilities and costs are not independent random variables, and the training examples for which costs are known are not representative of all examples.
Toward scalable learning with non-uniform class and cost distributions: a case study in credit card fraud detection
448 Citations1998Philip K. Chan, Salvatore J. Stolfo
A multi-classifier meta-learning approach to address very large databases with skewed class distributions and non-uniform cost per error and empirical results indicate that the approach can significantly reduce loss due to illegitimate transactions.
Lecture notes in computer scienceClass Imbalances versus Class Overlapping: An Analysis of a Learning System Behavior
341 Citations2004Ronaldo C. Prati, Gustavo Batista +1 more
This work develops a systematic study aiming to question whether class imbalances are truly to blame for the loss of performance of learning systems or whether the class imbalance are not a problem by themselves.
Class-Boundary Alignment for Imbalanced Dataset Learning
321 Citations2003Gang Wu, Edward Yi Chang
The class-boundaryalignment algorithm is proposed to augment SVMs to deal with imbalanced training-data problems posed by many emerging applications (e.g., image retrieval, video surveillance, and gene profiling).
ACM SIGKDD Explorations NewsletterExtreme re-balancing for SVMs
285 Citations2004Bhavani Raskutti, Adam Kowalczyk
There is a consistent pattern of performance differences between one and two-class learning for all SVMs investigated, and these patterns persist even with aggressive dimensionality reduction through automated feature selection.
Evaluating boosting algorithms to classify rare classes: comparison and improvements
271 Citations2002Manjunath V. Joshi, Vipin Kumar +1 more
The authors evaluate three existing categories of boosting algorithms from the single viewpoint of how they update the example weights in each iteration, and propose enhanced algorithms in two of the categories, and justify their choice of weight updating parameters theoretically.
Lecture notes in computer scienceConcept-Learning in the Presence of Between-Class and Within-Class Imbalances
135 Citations2001Nathalie Japkowicz
random re-sampling is extended in order to deal simultaneously with the between-class imbalance problem and addressing both problems simultaneously is beneficial and should be done by more sophisticated techniques as well.
2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03).On predicting rare classes with SVM ensembles in scene classification
129 Citations2003Rong Yan, Yan Liu +2 more
This paper proposes SVM ensembles to address the rare class problem and shows that hierarchical SVMs can achieve significantly better and more stable performance than other strategies, as well as high computational efficiency.
Learning classifiers from imbalanced data based on biased minimax probability machine
97 Citations2004Kaizhu Huang, Haiqin Yang +2 more
This paper proposes a novel model named biased minimax probability machine, which directly controls the worst-case real accuracy of classification of the future data to build up biased classifier and provides a rigorous treatment on imbalanced data.
Lecture notes in computer scienceA Mixture-of-Experts Framework for Learning from Imbalanced Data Sets
85 Citations2001Andrew Estabrooks, Nathalie Japkowicz
The proposed combination scheme is evaluated on a subset of the REUTERS-21578 text collection and is shown to be very effective when the data is drastically imbalanced.
Mixture of Expert Agents for Handling Imbalanced Data Sets
82 Citations2003Sotiris Kotsiantis, Panagiotis Pintelas
A systematic study on the various methodologies that have tried to handle skewed class distributions and an experimental study of these methodologies with a proposed mixture of expert agents conclude that such a framework can be a more effective solution to the problem.
Pattern Recognition LettersIncreasing sensitivity of preterm birth by changing rule strengths
36 Citations2002Jerzy W. Grzymala‐Busse, Linda Goodwin +1 more
The objective was to increase sensitivity by changing the rule strength for the preterm birth class by studying two prenatal data sets and two other medical data sets to choose the optimal rule strength.
Learning Rare Class Footprints: the REFLEX Algorithm
6 Citations2003Ray J. Hickey, Northern Ireland
Results of initial experiments comparing REFLEX to mestimation Laplace smoothing show that both algorithms deliver broadly similar performance for different contours.