Impact studies and sensitivity analysis in medical data mining with ROC-based genetic learning
Published 23 April 2004
Michèle Sébag, Jérôme Azé, Noël Lucas
Citations14
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is shown how a genetic algorithm-based optimization of the AUC criterion can be exploited for impact studies and sensitivity analysis on the Atherosclerosis Identification problem.
Abstract
ROC curves have been used for a fair comparison of machine learning algorithms since the late 90's. Accordingly, the area under the ROC curve (AUC) is nowadays considered a relevant learning criterion, accommodating imbalanced data, misclassification costs and noisy data. We show how a genetic algorithm-based optimization of the AUC criterion can be exploited for impact studies and sensitivity analysis. The approach is illustrated on the Atherosclerosis Identification problem, PKDD 2002 Challenge.
Keywords
Computer ScienceHealth Professions
The Nature of Statistical Learning Theory
39,279 Citations1995Vladimir Vapnik
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Pattern RecognitionThe use of the area under the ROC curve in the evaluation of machine learning algorithms
7,233 Citations1997Andrew P. Bradley
AUC exhibits a number of desirable properties when compared to overall accuracy: increased sensitivity in Analysis of Variance (ANOVA) tests; a standard error that decreased as both AUC and the number of test samples increased; decision threshold independent; and it is invariant to a priori class probabilities.
Oxford University Press eBooksEvolutionary Algorithms in Theory and Practice
4,299 Citations1996Thomas Bäck
In this work, the author compares the three most prominent representatives of evolutionary algorithms: genetic algorithms, evolution strategies, and evolutionary programming within a unified framework, thereby clarifying the similarities and differences of these methods.
Readings in Information Visualization: Using Vision to Think
3,967 Citations1999Stuart K. Card, Jock D. Mackinlay +1 more
The Annals of StatisticsBoosting the margin: a new explanation for the effectiveness of voting methods
2,313 Citations1998Peter L. Bartlett, Yoav Freund +2 more
It is shown that techniques used in the analysis of Vapnik's support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error.
Machine LearningA Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems
2,264 Citations2001David J. Hand, Robert John Till
This work extends the definition of the area under the ROC curve to the case of more than two classes by averaging pairwise comparisons and proposes an alternative definition of proportion correct based on pairwise comparison of classes for a simple artificial case.
MetaCost
1,294 Citations1999Pedro Domingos
A principled method for making an arbitrary classifier cost-sensitive by wrapping a cost-minimizing procedure around it is proposed, called MetaCost, which treats the underlying classifier as a black box, requiring no knowledge of its functioning or change to it.
The Annals of StatisticsArcing classifier (with discussion and a rejoinder by the author)
1,105 Citations1998Leo Breiman
Two arcing algorithms are explored, compared to each other and to bagging, and the definitions of bias and variance for a classifier as components of the test set error are introduced.
The Case against Accuracy Estimation for Comparing Induction Algorithms
1,024 Citations1998Foster Provost, Tom Fawcett +1 more
This work describes and demonstrates what it believes to be the proper use of ROC analysis for comparative studies in machine learning research, and argues that this methodology is preferable both for making practical choices and for drawing conclusions.
QUT ePrints (Queensland University of Technology)Boosting the margin: A new explanation for the effectiveness of voting methods
578 Citations1997Robert E. Schapire, Yoav Freund +2 more
Lecture notes in computer scienceAUC: A Better Measure than Accuracy in Comparing Learning Algorithms
449 Citations2003Charles X. Ling, Jin Huang +1 more
R rigourously is established that, even in this setting, the area under the ROC (Receiver Operating Characteristics) curve, or simply AUC, provides a better measure than accuracy when measuring and comparing classification systems.
Learning Decision Trees Using the Area Under the ROC Curve
266 Citations2002Cèsar Ferri, Peter Flach +1 more
This paper shows how a single decision tree can represent a set of classifiers by choosing different labellings of its leaves, or equivalently, an ordering on the leaves, and proposes a novel splitting criterion which chooses the split with the highest local AUC.
IEEE Transactions on Medical ImagingLinear and neural models for classifying breast masses
52 Citations1998David B. Fogel, Eugene C. Wasson +3 more
Results on 139 suspicious breast masses indicate that a significant probability of detecting malignancies can be achieved at the risk of a small percentage of false positives.
Lecture notes in computer scienceROC-Based Evolutionary Learning: Application to Medical Data Mining
27 Citations2004Michèle Sébag, Jérôme Azé +1 more
A novel way of comparing supervised learning algorithms has been introduced since the late 90's, based on Receiver Operating Characteristics (ROC) curves, which derived a NP complete optimization criterion for supervised learning, the area under the ROC curve.