Maximum profit mining and its application in software development
Published 20 August 2006
Charles X. Ling, Victor S. Sheng, Tilmann Bruckhaus, Nazim H. Madhavji
Citations20
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
An Escalation Prediction (EP) system that mines historic defect report data and predicts the escalation risk of the defects for maximum net profit to suggest that the cost-sensitive decision tree is the best method for producing the highest positive net profit and comprehensible results.
Abstract
While most software defects (i.e., bugs) are corrected and tested as part of the lengthy software development cycle, enterprise software vendors often have to release software products before all reported defects are corrected, due to deadlines and limited resources. A small number of these defects will be escalated by customers and they must be resolved immediately by the software vendors at a very high cost. In this paper, we develop an Escalation Prediction (EP) system that mines historic defect report data and predict the escalation risk of the defects for maximum net profit. More specifically, we first describe a simple and general framework to convert the maximum net profit problem to cost-sensitive learning. We then apply and compare several well-known cost-sensitive learning approaches for EP. Our experiments suggest that the cost-sensitive decision tree is the best method for producing the highest positive net profit and comprehensible results. The EP system has been deployed successfully in the product group of an enterprise software vendor.
Keywords
Computer Science
C4.5: Programs for Machine Learning
23,665 Citations1992J. R. Quinlan
A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.
The foundations of cost-sensitive learning
1,835 Citations2001Charles Elkan
It is argued that changing the balance of negative and positive training examples has little effect on the classifiers produced by standard Bayesian and decision tree learning methods, and the recommended way of applying one of these methods is to learn a classifier from the training set and then to compute optimal decisions explicitly using the probability estimates given by the classifier.
University of Maribor digital library (University of Maribor)Data Mining Techniques: For Marketing, Sales, and Customer Support
1,756 Citations1997Michael J. Berry, Gordon S. Linoff
One of the first practical guides to mining business data, Data Mining Techniques describes techniques for detecting customer behavior patterns useful in formulating marketing, sales, and customer support strategies.
MetaCost
1,294 Citations1999Pedro Domingos
A principled method for making an arbitrary classifier cost-sensitive by wrapping a cost-minimizing procedure around it is proposed, called MetaCost, which treats the underlying classifier as a black box, requiring no knowledge of its functioning or change to it.
Journal of Artificial Intelligence ResearchLearning When Training Data are Costly: The Effect of Class Distribution on Tree Induction
928 Citations2003Gary M. Weiss, Foster Provost
A "budget-sensitive" progressive sampling algorithm is introduced for selecting training examples based on the class associated with each example and it is shown that the class distribution of the resulting training set yields classifiers with good (nearly-optimal) classification performance.
C4.5, Class Imbalance, and Cost Sensitivity: Why Under-Sampling beats Over-Sampling
833 Citations2003Chris Drummond, Robert C. Holte
This paper shows that using C4.5 with undersampling establishes a reasonable standard for algorithmic comparison, and it is recommended that the cheapest class classifier be part of that standard as it can be better than under-sampling for relatively modest costs.
ComputerTop 10 list [software development]
796 Citations2001Barry Boehm, Victor R. Basili
Reducing “ avoidable rework” is a major source of software productivity improvement and most effort savings from improving software processes, architectures and risk management are results of reductions in “avoidable re work”.
Cost-sensitive learning by cost-proportionate example weighting
669 Citations2004Bianca Zadrozny, John Langford +1 more
Costing is proposed, a method based on cost-proportionate rejection sampling and ensemble aggregation, which achieves excellent predictive performance on two publicly available datasets, while drastically reducing the computation required by other methods.
Journal of Artificial Intelligence ResearchCost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm
576 Citations1995Peter D. Turney
This paper introduces ICET, a new algorithm for cost-sensitive classification that uses a genetic algorithm to evolve a population of biases for a decision tree induction algorithm and establishes that ICET performs significantly better than its competitors.
IEEE Transactions on Knowledge and Data EngineeringAn instance-weighting method to induce cost-sensitive trees
491 Citations2002Kai Ming Ting
The algorithm incorporating the instance-weighting method is found to be better than the original algorithm in in of total misclassification costs, the number of high cost errors, and tree size two-class data sets.
Software Engineering Economics
434 Citations2001Barry Boehm
Elsevier eBooksInduction of One-Level Decision Trees
359 Citations1992Wayne Iba, Pat Langley
This paper presents an average-case analysis of a simple algorithm that induces one-level decision trees for concepts defined by a single relevant attribute, and derives the expected classification accuracy over the entire instance space.
Software engineering economics
355 Citations2002Hakan Erdogmus, Barry Boehm +3 more
This tutorial is meant in part to enable those who would like to participate in the workshop, but who might not possess the requisite background, to come up to speed and is designed to be self-contained and will cover concepts from introductory to advanced.
Decision trees with minimal costs
278 Citations2004Charles X. Ling, Qiang Yang +2 more
A simple, novel and yet effective method for building and testing decision trees that minimizes the sum of the misclassification and test costs and design several intelligent test strategies that can suggest ways of obtaining the missing values at a cost in order to minimize the total cost.
HAL (Le Centre pour la Communication Scientifique Directe)Proceedings of the 2010 International Conference on Data Mining
245 Citations2010Robert Stahlbock, Sven F. Crone +6 more
Learning when training data are costly
35 Citations2003M WeissGary, ProvostFoster
Cost-sensitive classification
19 Citations1995D TurneyPeter
Knowledge Discovery and Data MiningThe Business Impact of Predictive Analytics
15 Citations2007Tilmann Bruckhaus
This chapter will illustrate the use of technical and financial business impact metrics with running examples in the areas of insurance fraud prevention and predictive marketing.
Predicting Software Escalations with Maximum ROI
14 Citations2006Charles X. Ling, Shengli Sheng +2 more
This paper develops an Escalation Prediction (EP) system to mine historic defect report data and predict the escalation risk of current defect reports for maximum ROI (Return On Investment).
Software Escalation Prediction with Data Mining
13 Citations2004Tilmann Bruckhaus, Charles X. Ling +2 more
The business case for EP is outlined, an analysis of the business problem, the solution architecture, and some preliminary validation results on the effectiveness of EP are outlined, to avoid escalations from known product defects by predicting and proactively resolving those known defects that have the highest escalation risk.