Error Rate Estimation in Discriminant Analysis: Recent Advances
Published 1 January 1987
Geoffrey J. McLachlan
Citations26
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
An important problem in discriminant analysis is the estimation of the error rates associated with a given discriminant rule for allocating an object of unknown origin to one of a finite number, say g, of distinct classes or populations. The rule is based on the observed value of a random vector X of p measurements on the object. Over the years there have been many investigations on this problem; see, for example, Hills (1966), Lachenbruch and Mickey (1968), and McLachlan (1974a, b, c), and the references therein. Toussaint (1974) has compiled an extensive bibliography, which has been updated recently by Hand (1986b). An overview of error rate estimation has been given by McLachlan (1986), while recent work on robust error rate estimation has been summarized by Knoke (1986).
Keywords
ChemistryMathematics
The Annals of StatisticsBootstrap Methods: Another Look at the Jackknife
17,446 Citations1979B. Efron
Journal of Business and Economic StatisticsAn Introduction to Multivariate Statistical Analysis
9,232 Citations1986Robb J. Muirhead, T. W. Anderson
Society for Industrial and Applied Mathematics eBooksThe Jackknife, the Bootstrap and Other Resampling Plans
7,903 Citations1982Bradley Efron
Journal of the American Statistical AssociationEstimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation
2,161 Citations1983Bradley Efron
TechnometricsEstimation of Error Rates in Discriminant Analysis
1,495 Citations1968Peter A. Lachenbruch, M. R. Mickey
Several methods of estimating error rates in Discriminant Analysis are evaluated by sampling methods, finding the two methods in most common use to be significantly poorer than some new methods that are proposed.
Journal of the American Statistical AssociationHow Biased is the Apparent Error Rate of a Prediction Rule?
419 Citations1986Bradley Efron
IEEE Transactions on Information TheoryBibliography on estimation of misclassification
307 Citations1974Godfried Toussaint
Articles, books, and technical reports on the theoretical and experimental estimation of probability of misclassification are listed for the case of correctly labeled or preclassified training data.
Journal of the American Statistical AssociationCross-Validation, the Jackknife, and the Bootstrap: Excess Error Estimation in Forward Logistic Regression
236 Citations1986Gail Gong
Three estimates of the excess error are considered: cross-validation, the jackknife, and the bootstrap, for a moderately complicated prediction rule, involving a variable-selection procedure based on forward logistic regression.
Journal of the Royal Statistical Society Series B (Statistical Methodology)Allocation Rules and Their Error Rates
216 Citations1966Michael Hills
Journal of the American Statistical AssociationDiscrimination and Classification Using Both Binary and Continuous Variables
205 Citations1975W. J. Krzanowski
A method is proposed for estimating the likelihood ratio classification rule in practical situations and assessing its performance, and its performance is compared with that of some other classification rules.
Pattern RecognitionAdditive estimators for probabilities of correct classification
146 Citations1978Ned Glick
“Smooth” modification of the counting estimator, or sample success proportion, is recommended to reduce bias and variance while retaining robustness, and the “bootstrap” method of Efron can approximately correct an additive estimator's bias using an ancillary computer simulation.
Pattern Recognition LettersRecent advances in error rate estimation
132 Citations1986David J. Hand
Different types of error rates are described and the state of the art of error rate estimation at the time of Toussaint's survey is briefly summarised, and the two major advances, namely bootstrap and average conditional error rates estimation methods, and their extensions, are described in detail.
IEEE Transactions on Information TheoryNonparametric Bayes error estimation using unclassified samples
88 Citations1973Keinosuke Fukunaga, David Kessell
A pattern recognition system may be viewed as a decision rule which transforms measurements into class assignments and the Bayes error is the minimum achievable error, where the minimization is with respect to all decision rules.
BiometricsAn Asymptotic Unbiased Technique for Estimating the Error Rates in Discriminant Analysis
71 Citations1974Geoffrey J. McLachlan
Pattern Recognition LettersApplication of bootstrap and other resampling techniques: Evaluation of classifier performance
61 Citations1985Michael R. Chernick, V. K. Murthy +1 more
This paper presents a simulation comparing various resampling procedures for estimating classification error rate, for the two-class and three-class problems.
BiometrikaThe bias of the apparent error rate in discriminant analysis
51 Citations1976Geoffrey J. McLachlan
Journal of Marketing ResearchEstimation of Error Rates in Several-Population Discriminant Analysis
48 Citations1982Stephen C. Hora, James B. Wilcox
TechnometricsEstimation of the Errors of Misclassification on the Criterion of Asymptotic Mean Square Error
43 Citations1974Geoffrey J. McLachlan
TechnometricsAn Evaluation of Smoothed Classification Error – Rate Estimators
43 Citations1985Steven Snapinn, James D. Knoke
Using the criterion of unconditional mean squared error, this particular smoothed estimator, called the NS method, appears to offer a reasonable alternative to existing nonparametric estimators.
Computers in Biology and MedicineAn efficient method for estimating the probability of misclassification applied to a problem in medical diagnosis
35 Citations1975Godfried T. Toussaint, Phil Sharpe
A new method of estimating the probability of misclassification is proposed which yields essentially unbiased results similar to Lachenbruch's U -method with far less computation involved.
TechnometricsClassification Error Rate Estimators Evaluated by Unconditional Mean Squared Error
34 Citations1984Steven Snapinn, James D. Knoke
Communications in Statistics - Simulation and ComputationEstimation of misclassification probabilities by bootstrap methods
30 Citations1983Samprit Chatterjee, Sangit Chätterjee
The application of bootstrap sampling to the above problem has the advantage of not only furnishing the estimates of misclassification probabilities but also provides an estimate of the standard error of estimate.
Australian Journal of StatisticsAN ASYMPTOTIC EXPANSION OF THE EXPECTATION OF THE ESTIMATED ERROR RATE IN DISCRIMINANT ANALYSIS1
29 Citations1973Geoffrey J. McLachlan
Computers & Mathematics with ApplicationsAssessing the performance of an allocation rule
29 Citations1986Geoffrey J. McLachlan
Attention is focussed on the formation of improved estimates, mainly through appropriate bias correction of the apparent error rate, and the role of the bootstrap, a computer-based methodology, is highlighted.
Journal of the American Statistical AssociationEstimation of Allocation Rates in a Cluster Analysis Context
25 Citations1985K. E. Basford, Geoffrey J. McLachlan
Pattern RecognitionA note on the choice of a weighting function to give an efficient method for estimating the probability of misclassification
24 Citations1977Geoffrey J. McLachlan
In an attempt to provide a nonparametric estimator which not only uses the data efficiently but is also essentially an unbiased estimator of the probability of misclassification, Toussaint evaluated empirically an estimator formed by weighting the resubstitution and rotation estimators.
Journal of Statistical Computation and SimulationThe efficiency of Efron's “Bootstrap” Approach Applied to Error Rate Estimation in Discriminant Analysis
22 Citations1980Geoffrey J. McLachlan
Journal of the American Statistical AssociationEstimation of Allocation Rates in a Cluster Analysis Context
21 Citations1985K. E. Basford, Geoffrey J. McLachlan
TechnometricsThe Relationship in Terms of Asymptotic Mean Square Error Between the Separate Problems of Estimating each of the Three Types of Error Rate of the Linear Discriminant Function
19 Citations1974Geoffrey J. McLachlan
Pattern RecognitionError rate estimation on the basis of posterior probabilities
17 Citations1980S. Gànesalingam, Geoffrey J. McLachlan
The so-called posterior probability estimator, e, formed by averaging the minimum of the posterior probabilities over a set of initial or additional observations (which need not be classified) is considered in the context of estimating the overall actual error rate for the linear discriminant function appropriate for two multivariate normal populations with a common covariance matrix.
Pattern RecognitionSmoothed additive estimators for non-error rates in multiple discriminant analysis
14 Citations1985Gerhard Tutz
In simulation experiments with normal, uniform and discrete distributions the smoothed additive estimators with fixed and variable smoothing parameter are compared to the leaving-one out method and the resubstitution method with respect to bias and variance.
Computers & Mathematics with ApplicationsEstimation of error rate for linear discriminant functions by resampling: Non-Gaussian populations
13 Citations1988Michael R. Chernick, V. K. Murthy +1 more
Simulation results comparing various resampling estimators of classification error rate for linear discriminant type classification algorithms for three non-Gaussian multivariate populations, namely, exponential, Cauchy and uniform are presented.
Computers & Mathematics with ApplicationsThe robust estimation of classification error rates
11 Citations1986James D. Knoke
Recommendations for applications based on available evaluations of robust estimators are made, and important unresolved issues are identified.
Biometrical JournalComparison of various procedures for estimation of the classification error in discriminance analysis
10 Citations1980K.‐D. Wernecke, G. Kalb +1 more
It is shown by means of a practical example in which way the number of characteristics used in the discrimination affects the classification error, and recommendations for the selection of practicable estimation methods are given.
Computational Statistics & Data AnalysisRe-sampling procedures for reducing bias of error rate estimation in multinomial classification
8 Citations1986Min-Chiang Wang
Cross-validation, jackknife and bootstrap procedures are introduced to obtain the bias-reducing estimates of error rate and it is found that all three procedures are effective in reducing bias under small sample situations.
Communications in Statistics - Simulation and ComputationThe jackknife procedure for the probabilities of misclassification
7 Citations1985Poduri S.R.S. Rao, Atsu Shelter Dorvlo
Communications in Statistics - Simulation and ComputationPosterior probability estimators in classification simulations
7 Citations1980Gregory T. Schwemer, Olive Jean Dunn
Biometrical JournalFurther Results in Estimating the Classification Error in Discriminance Analysis
6 Citations1983K.‐D. Wernecke, G. Kalb
Computers & Mathematics with ApplicationsOn the estimation of the expected probability of misclassification in discriminant analysis with mixed binary and continuous variables
6 Citations1986Ioannis G. Vlachonikolis
Nonparametric Bayes error estimation using unclassified samples
3 Citations1972Keinosuke Fukunaga, David Kessell
Pattern Recognition LettersThe asymptotic distribution of an estimator of the Bayes error rate
2 Citations1984Norman Matloff, Ronald C. Pruitt
The asymptotic distribution of a class of estimators of the Bayes error rate in pattern recognition problems is derived from estimator of the risk-averaging type, applied to parametric models such as the logistic function.