Cost-sensitive learning and decision making revisited
European Journal of Operational ResearchPublished 25 June 2004
Stijn Viaene, Guido Dedene
Citations64
SJR quartileQ1
SJR score2.24
SNIP2.62
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper brings an up-to-date overview of several methods that aim to make a broad variety of error-based learners cost-sensitive, including direct minimum expected cost classification, MetaCost, over- and undersampling, and cost- sensitive boosting.
Abstract
In many real-life decision making situations the default assumption of equal misclassification costs underlying pattern recognition techniques is most likely violated. Then, cost-sensitive learning and decision making bring help for making cost-benefit-wise optimal decisions. This paper brings an up-to-date overview of several methods that aim to make a broad variety of error-based learners cost-sensitive. More specifically, we revisit direct minimum expected cost classification, MetaCost, over- and undersampling, and cost-sensitive boosting. (c) 2004 Elsevier B.V. All rights reserved.
Keywords
Computer Science
C4.5: Programs for Machine Learning
23,665 Citations1992J. R. Quinlan
A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.
RadiologyThe meaning and use of the area under a receiver operating characteristic (ROC) curve.
21,812 Citations1982J A Hanley, Barbara J. McNeil
A representation and interpretation of the area under a receiver operating characteristic (ROC) curve obtained by the "rating" method, or by mathematical predictions based on patient characteristics, is presented and it is shown that in such a setting the area represents the probability that a randomly chosen diseased subject is (correctly) rated or ranked with greater suspicion than a random chosen non-diseased subject.
Journal of Computer and System SciencesA Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
20,320 Citations1997Yoav Freund, Robert E. Schapire
Machine LearningBagging predictors
16,377 Citations1996Leo Breiman
Tests on real and simulated data sets using classification and regression trees and subset selection in linear regression show that bagging can give substantial gains in accuracy.
Advances in industrial controlPattern Classification
7,884 Citations2000Evan L. Russell, Leo H. Chiang +1 more
Pattern RecognitionThe use of the area under the ROC curve in the evaluation of machine learning algorithms
7,233 Citations1997Andrew P. Bradley
AUC exhibits a number of desirable properties when compared to overall accuracy: increased sensitivity in Analysis of Variance (ANOVA) tests; a standard error that decreased as both AUC and the number of test samples increased; decision threshold independent; and it is invariant to a priori class probabilities.
The Annals of StatisticsAdditive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors)
6,842 Citations2000Jerome H. Friedman, Trevor Hastie +1 more
This work shows that this seemingly mysterious phenomenon of boosting can be understood in terms of well-known statistical principles, namely additive modeling and maximum likelihood, and develops more direct approximations and shows that they exhibit nearly identical results to boosting.
Journal of Artificial Intelligence ResearchPopular Ensemble Methods: An Empirical Study
2,972 Citations1999David W. Opitz, Richard Maclin
This work suggests that most of the gain in an ensemble's performance comes in the first few classifiers combined; however, relatively large gains can be seen up to 25 classifiers when Boosting decision trees.
Machine LearningAn Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants
2,644 Citations1999Eric Bauer, Ron Kohavi
It is found that Bagging improves when probabilistic estimates in conjunction with no-pruning are used, as well as when the data was backfit, and that Arc-x4 behaves differently than AdaBoost if reweighting is used instead of resampling, indicating a fundamental difference.
Improved boosting algorithms using confidence-rated predictions
2,568 Citations1998Robert E. Schapire, Yoram Singer
The Annals of StatisticsBoosting the margin: a new explanation for the effectiveness of voting methods
2,313 Citations1998Peter L. Bartlett, Yoav Freund +2 more
It is shown that techniques used in the analysis of Vapnik's support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error.
Machine LearningA Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems
2,264 Citations2001David J. Hand, Robert John Till
This work extends the definition of the area under the ROC curve to the case of more than two classes by averaging pairwise comparisons and proposes an alternative definition of proportion correct based on pairwise comparison of classes for a simple artificial case.
Lecture notes in statisticsThe Boosting Approach to Machine Learning: An Overview
2,013 Citations2003Robert E. Schapire
This chapter overviews some of the recent work on boosting including analyses of AdaBoost's training error and generalization error; boosting’s connection to game theory and linear programming; the relationship between boosting and logistic regression; extensions of Ada boost for multiclass classification problems; methods of incorporating human knowledge into boosting; and experimental and applied work using boosting.
Machine LearningImproved Boosting Algorithms Using Confidence-rated Predictions
1,959 Citations1999Robert E. Schapire, Yoram Singer
Several improvements to Freund and Schapire's AdaBoost boosting algorithm are described, particularly in a setting in which hypotheses may assign confidences to each of their predictions.
The foundations of cost-sensitive learning
1,835 Citations2001Charles Elkan
It is argued that changing the balance of negative and positive training examples has little effect on the classifiers produced by standard Bayesian and decision tree learning methods, and the recommended way of applying one of these methods is to learn a classifier from the training set and then to compute optimal decisions explicitly using the probability estimates given by the classifier.
MetaCost
1,294 Citations1999Pedro Domingos
A principled method for making an arbitrary classifier cost-sensitive by wrapping a cost-minimizing procedure around it is proposed, called MetaCost, which treats the underlying classifier as a black box, requiring no knowledge of its functioning or change to it.
Machine LearningRobust Classification for Imprecise Environments
1,206 Citations2001Foster Provost, Tom Fawcett
It is shown that it is possible to build a hybrid classifier that will perform at least as well as the best available classifier for any target conditions, and in some cases, the performance of the hybrid actually can surpass that of the best known classifier.
The Case against Accuracy Estimation for Comparing Induction Algorithms
1,024 Citations1998Foster Provost, Tom Fawcett +1 more
This work describes and demonstrates what it believes to be the proper use of ROC analysis for comparative studies in machine learning research, and argues that this methodology is preferable both for making practical choices and for drawing conclusions.
Machine LearningMultiBoosting: A Technique for Combining Boosting and Wagging
699 Citations2000Geoffrey I. Webb
MultiBoosting is an extension to the highly successful AdaBoost technique for forming decision committees that is able to harness both AdaBoost's high bias and variance reduction with wagging's superior variance reduction.
Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers
656 Citations2001Bianca Zadrozny, Charles Elkan
It is concluded that binning succeeds in significantly improving naive Bayesian probability estimates, while for improving decision tree probability estimates the recommend smoothing by -estimation and a new variant of pruning that is called curtailment.
TechnometricsConstruction and Assessment of Classification Rules
625 Citations1999Mark R. Wade, David J. Hand
Basic Ideas Introduction Constructing Rules Fisher's LDA and Other Methods Based on Covariance Matrices Recursive Partitioning Methods Nonparametric Smoothing Methods Practical Issues Some Special Problems Some Illustrative Applications.
AdaCost: Misclassification Cost-Sensitive Boosting
596 Citations1999Wei Fan, Salvatore J. Stolfo +2 more
It is formally show that AdaCost reduces the upper bound of cumulative misclassification cost of the training set of the training set without consuming additional computing power.
Learning and making decisions when costs and probabilities are both unknown
455 Citations2001Bianca Zadrozny, Charles Elkan
This work explains how to obtain unbiased estimators for example-dependent costs, taking into account the difficulty that in general, probabilities and costs are not independent random variables, and the training examples for which costs are known are not representative of all examples.
European Journal of Operational ResearchBayesian neural network learning for repeat purchase modelling in direct marketing
216 Citations2002Bart Baesens, Stijn Viaene +3 more
Experimental evidence is provided that Bayesian neural networks offer a viable alternative for purchase incidence modelling and a combined use of all three RFM predictor categories is advocated by the ARD method.
Lecture notes in computer sciencePruning decision trees with misclassification costs
197 Citations1998Jeffrey P. Bradford, Clayton Kunz +3 more
It was found that applying the Laplace correction to estimate the probability distributions at the leaves was beneficial to all pruning methods and led to results that were on par with other methods in terms of the evaluation criteria.
Explicitly representing expected cost
186 Citations2000Chris Drummond, Robert C. Holte
There is a point/line duality between the two representations of ROC representation, allowing most techniques used in ROC analysis to be readily reproduced in the cost space.
Springer texts in statisticsClassification and Regression Trees (CART)
85 Citations2020Richard A. Berk
Methods for cost-sensitive learning
42 Citations2002Thomas G. Dietterich, Dragos D. Margineantu
B-LOTs were shown to be superior to other methods in cases where the classes have very different frequencies—a situation that arises frequently in cost-sensitive classification problems.
Lecture notes in computer scienceBayes and pseudo-Bayes estimates of conditional probabilities and their reliability
35 Citations1993James Cussens
Various ways of estimating probabilities, mainly within the Bayesian framework, are discussed and their relevance and application to machine learning is given, and their relative performance empirically evaluated.
Lecture notes in computer scienceImproving the Performance of Boosting for Naive Bayesian Classification
34 Citations1999Kai Ming Ting, Zijian Zheng
The experimental results show that although introducing tree structures into naive Bayesian classification increases the average error of naiveBayesian classification for individual models, boosting naïve Bayesian classifiers with tree structures can achieve significantly lower average error than the naive Bayesesian classifier.