Adaptive Feature Spaces for Land Cover Classification with Limited Ground Truth Data
Lecture notes in computer sciencePublished 1 January 2002
Joseph Troy Morgan, Alex Henneguelle, Melba M. Crawford, Joydeep Ghosh, Amy Neuenschwander
Citations13
SJR quartileQ2
SJR score0.35
SNIP0.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The resulting best-basis binary hierarchical classifier (BB-BHC) family is thus able to address the "small sample size" problem, as evidenced by the experimental results.
Abstract
Classification of hyperspectral data is challenging because of high dimensionality (O(100)) inputs, several possible output classes with uneven priors, and scarcity of labeled information. In an earlier work, a multiclassifier system arranged as a binary hierarchy was developed to group classes for easier, progressive discrimination [27]. This paper substantially expands the scope of such a system by integrating a feature reduction scheme that adaptively adjusts to the amount of labeled data available, while exploiting the highly correlated nature of certain adjacent hyperspectral bands. The resulting best-basis binary hierarchical classifier (BB-BHC) family is thus able to address the "small sample size" problem, as evidenced by our experimental results.
Keywords
ChemistryComputer ScienceEngineering
Machine LearningRandom Forests
126,192 Citations2001Leo Breiman
Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the forest, and are also applicable to regression.
The Nature of Statistical Learning Theory
39,279 Citations1995Vladimir Vapnik
Choice Reviews OnlineNeural networks for pattern recognition
18,687 Citations1994
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Elsevier eBooksIntroduction to Statistical Pattern Recognition
11,121 Citations1990Keinosuke Fukunaga
A system to determine the sex of fish from their length, as accurately as possible, using a set of labelled data, together with the class label (male or female), as a training set.
Journal of Business and Economic StatisticsAn Introduction to Multivariate Statistical Analysis
9,232 Citations1986Robb J. Muirhead, T. W. Anderson
IEEE Transactions on Pattern Analysis and Machine IntelligenceStatistical pattern recognition: a review
6,688 Citations2000Anil K. Jain, Peter Duin +1 more
The objective of this review paper is to summarize and compare some of the well-known methods used in various stages of a pattern recognition system and identify research topics and applications which are at the forefront of this exciting and challenging field.
Combining labeled and unlabeled data with co-training
5,604 Citations1998Avrim Blum, Tom M. Mitchell
Journal of Artificial Intelligence ResearchSolving Multiclass Learning Problems via Error-Correcting Output Codes
2,718 Citations1995Tom Dietterich, Ghulum Bakiri
It is demonstrated that error-correcting output codes provide a general-purpose method for improving the performance of inductive learning programs on multiclass problems.
Journal of the American Statistical AssociationRegularized Discriminant Analysis
2,238 Citations1989Jerome H. Friedman
Alternatives to the usual maximum likelihood estimates for the covariance matrices are proposed, characterized by two parameters, the values of which are customized to individual situations by jointly minimizing a sample-based estimate of future misclassification risk.
Statistical Pattern Recognition
1,754 Citations2001
Large Margin DAGs for Multiclass Classification
1,644 Citations1999John Platt, Nello Cristianini +1 more
An algorithm, DAGSVM, is presented, which operates in a kernel-induced feature space and uses two-class maximal margin hyperplanes at each decision-node of the DDAG, which is substantially faster to train and evaluate than either the standard algorithm or Max Wins, while maintaining comparable accuracy to both of these algorithms.
IEEE Transactions on Pattern Analysis and Machine IntelligenceSmall sample size effects in statistical pattern recognition: recommendations for practitioners
1,366 Citations1991Šarūnas Raudys, Anil K. Jain
The effects of sample size on feature selection and error estimation for several types of classifiers are discussed and an emphasis is placed on giving practical advice to designers and users of statistical pattern recognition systems.
The Annals of StatisticsClassification by pairwise coupling
1,291 Citations1998Trevor Hastie, Robert Tibshirani
A strategy for polychotomous classification that involves estimating class probabilities for each pair of classes, and then coupling the estimates together is discussed, similar to the Bradley-Terry method for paired comparisons.
IEEE Signal Processing MagazineHyperspectral image data analysis
1,102 Citations2002D. A. Landgrebe
The article includes an example of an image space representation, using three bands to simulate a color IR photograph of an airborne hyperspectral data set over the Washington, DC, mall.
Neural ComputationNeural Network Classifiers Estimate Bayesian <i>a posteriori</i> Probabilities
993 Citations1991Michael D. Richard, Richard P. Lippmann
Results of Monte Carlo simulations performed using multilayer perceptron (MLP) networks trained with backpropagation, radial basis function (RBF) networks, and high-order polynomial networks graphically demonstrate that network outputs provide good estimates of Bayesian probabilities.
Machine LearningOn the Learnability and Design of Output Codes for Multiclass Problems
660 Citations2002Koby Crammer, Yoram Singer
This paper discusses for the first time the problem of designing output codes for multiclass problems, and gives a time and space efficient algorithm for solving the quadratic program.
Connection ScienceError Correlation and Error Reduction in Ensemble Classifiers
602 Citations1996Kagan Tumer, Joydeep Ghosh
This paper focuses on data selection and classifier training methods, in order to 'prepare' classifiers for combining, and discusses several methods that make the classifiers in an ensemble more complementary.
IEEE Transactions on Geoscience and Remote SensingThe effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon
572 Citations1994B.M. Shahshahani, D. A. Landgrebe
By using additional unlabeled samples that are available at no extra cost, the performance may be improved, and therefore the Hughes phenomenon can be mitigated and therefore more representative estimates can be obtained.
Improving Text Classification by Shrinkage in a Hierarchy of Classes
478 Citations2022Andrew McCallum, Roni Rosenfeld +2 more
This paper shows that the accuracy of a naive Bayes text classi(cid:12)er can be significantly improved by taking advantage of a hierarchy of classes, and adopts an established statistical technique called shrinkage that smoothes parameter estimates of a data-sparse child with its parent in order to obtain more robust parameter estimates.
IEEE Transactions on Neural NetworksEfficient classification for multiclass problems using modular neural networks
396 Citations1995Rangachari Anand, K. Mehrotra +2 more
The approach is to use a modular network architecture, reducing a K-class problem to a set of K two-class problems, with a separately trained network for each of the simpler problems.
IEEE Transactions on Geoscience and Remote SensingSegmented principal components transformation for efficient hyperspectral remote-sensing image display and classification
394 Citations1999Xiuping Jia, John A. Richards
A segmented, and possibly multistage, principal components transformation (PCT) is proposed for efficient hyperspectral remote-sensing image classification and display and results have been obtained in terms of classification accuracy, speed, and quality of color image display using two airborne visible/infrared imaging spectrometer (AVIRIS) data sets.
IEEE Transactions on Pattern Analysis and Machine IntelligenceFeature extraction based on decision boundaries
376 Citations1993C. Lee, D. A. Landgrebe
The proposed feature extraction algorithm has several desirable properties: it predicts the minimum number of features necessary to achieve the same classification accuracy as in the original space for a given pattern recognition problem; and it finds the necessary feature vectors.
Lecture notes in computer scienceTransformation Invariance in Pattern Recognition – Tangent Distance and Tangent Propagation
348 Citations2012Patrice Y. Simard, Yann LeCun +2 more
IEEE Transactions on Geoscience and Remote SensingBest-bases feature extraction algorithms for classification of hyperspectral data
244 Citations2001Shailesh Kumar, Joydeep Ghosh +1 more
A set of best-bases feature extraction algorithms that are simple, fast, and highly effective for classification of hyperspectral data are proposed.
Pattern Recognition Theory and Applications
206 Citations1987Pierre A. Devijver, Josef Kittler
Current research emphasis in pattern recognition is on designing efficient algorithms, studying small sample properties of various estimators and decision rules, implementing the algorithms on novel computer architecture, and incorporating context and domain-specific knowledge in decision making.
Learning Classification with Unlabeled Data
204 Citations1993Virginia R. de
This paper shows that minimizing the disagreement between the outputs of networks processing patterns from these different modalities is a sensible approximation to minimizing the number of misclassifications in each modality, and leads to similar results.
Pattern Analysis and ApplicationsHierarchical Fusion of Multiple Classifiers for Hyperspectral Data Analysis
188 Citations2002Shailesh Kumar, Joydeep Ghosh +1 more
This paper introduces a hierarchical technique to recursively decompose a C-class problem into C_1 two-(meta) class problems, and introduces a generalised modular learning framework used to partition a set of classes into two disjoint groups called meta-classes.
Pattern Recognition LettersExpected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix
185 Citations1998Šarūnas Raudys, Robert P. W. Duin
An asymptotic formula for the expected (generalization) error of the Fisher classifier with the pseudo-inversion is derived which explains the peaking behaviour: with an increasing number of learning observations from one up to the number of features, the generalization error first decreases, and then starts to increase.
Studies in fuzziness and soft computingStatistical pattern recognition
182 Citations2000Ludmila I. Kuncheva
Pattern recognition is about those problems that are still not algorithmically clear-cut, but which humans easily identify faces, speakers, smells — tasks that are still a challenge for a computer.
IEEE Transactions on Geoscience and Remote SensingAn adaptive classifier design for high-dimensional data analysis with a limited training data set
172 Citations2001Q. Jackson, D. A. Landgrebe
Experimental results show this proposed adaptive classifier can improve the classification accuracy as well as representation of estimated statistics significantly, and therefore the Hughes phenomenon may be mitigated.
IEEE Transactions on Geoscience and Remote SensingCovariance estimation with limited training samples
167 Citations1999S. Tadjudin, D. A. Landgrebe
A covariance estimator formulated under an empirical Bayesian setting to mitigate the problem of limited training samples in the Gaussian maximum likelihood (ML) classification for remote sensing is described.
International Journal of Imaging Systems and TechnologyTransformation invariance in pattern recognition: Tangent distance and propagation
150 Citations2000Patrice Y. Simard, Yann A. Le Cun +2 more
The concept of tangent vectors is introduced, which compactly represent the essence of these transformation invariances, and two classes of algorithms, tangent distance and tangent propagation, which make use of theseinvariances to improve performance.
IEEE Transactions on Neural NetworksDecision boundary feature extraction for neural networks
137 Citations1997Chulhee Lee, D. A. Landgrebe
IEEE Transactions on Geoscience and Remote SensingRobust parameter estimation for mixture model
119 Citations2000S. Tadjudin, D. A. Landgrebe
Hyperspectral Image Data Analysis as a High Dimensional Signal Processing Problem
117 Citations2002D. A. Landgrebe
It was the possibility of spacecraft, pattern recognition technology, and the digital computer that stimulated thought into how one might make observations from space to obtain information in order to better manage the Earth’s renewable and nonrenewable resources.
IEEE Transactions on Geoscience and Remote SensingPartially supervised classification using weighted unsupervised clustering
67 Citations1999Byeungwoo Jeon, D. A. Landgrebe
This paper addresses a classification problem in which class definition through training samples or otherwise is provided a priori only for a particular class of interest, and a conventional supervised classifier such as the maximum likelihood classifier is used in the classification.
Stabilizing classifiers for very small sample sizes
67 Citations1996Marina Skurichina, Robert P. W. Duin
A stability measure is proposed and a study on the performance and stability of the following techniques: regularization by the ridge-estimate of the covariance matrix, bootstrapping followed by aggregation and editing combined with pseudo-inversion are presented.
Advances in Statistical Pattern Recognition
53 Citations1987Anil K. Jain
Current research emphasis in pattern recognition is on designing efficient algorithms, studying small sample properties of various estimators and decision rules, implementing the algorithms on novel computer architecture, and incorporating context and domain-specific knowledge in decision making.
Decimated input ensembles for improved generalization
45 Citations2003Kagan Turner, Nikunj C. Oza
The need to reduce the correlation among the component classifiers is highlighted and one method for correlation reduction is investigated: input decimation, a method that uses the discriminating features of the inputs to decouple classifiers.
Information FusionBinary labelling and decision-level fusion
41 Citations2001Terry Windeatt, Reza Ghaderi
Two binary labelling techniques for decision-level fusion are considered for reducing correlation in the context of multiple classifier systems and a method based on error correcting coding that uses binary code words to decompose a multi-class problem into a set of complementary two-class problems is described.
Wiley Encyclopedia of Electrical and Electronics EngineeringInformation Processing for Remote Sensing
38 Citations1999James C. Tilton, D. A. Landgrebe +1 more
The sections in this article are Feature Extraction, Multispectral Image Data Classification, Classification Using Neural Networks, and Hyperspectral Data.
Lecture notes in computer scienceA Hierarchical Multiclassifier System for Hyperspectral Data Analysis
30 Citations2000Shailesh Kumar, Joydeep Ghosh +1 more
A hierarchical multiclassifier paradigm in which a C- class problem is recursively decomposed into C - 1 two-class problems is introduced, which was used to classify 12 types of landcover from 183-dimensional hyperspectral data and showed the potential of such a modular learning approach for discovering domain knowledge automatically from data.
IEEE Transactions on Neural NetworksBackpropagation uses prior information efficiently
28 Citations1993Etienne Barnard, Elizabeth C. Botha
Backpropagation employs a priori information in a slightly suboptimal fashion, but this does not have serious consequences on the performance of the classifier and the inferior generalization that results when an excessive number of network parameters are used can be ascribed to this suboptimality.
A versatile framework for labelling imagery with a large number of classes
18 Citations2003Shailesh Kumar, Melba M. Crawford +1 more
This work proposes a class-pair based feature selection algorithm which, in conjunction with mixture modeling technique, provides significantly superior results for differentiating a large number of classes, even when the class priors vary considerably.
Modular learning through output space decomposition
18 Citations2000Shailesh Kumar, Joydeep Ghosh +1 more
The modular learning paradigm is extended to the previously unexplored, but potentially useful, problem decomposition framework in which a C > 2 class problem is systematically decomposed into simpler two-(meta)class problems.
Journal of Intelligent Material Systems and StructuresThe Rapid Kernel Classifier: A Link between the Self-Organizing Feature Map and the Radial Basis Function Network
11 Citations1994Joydeep Ghosh, V. Srinivasa Chakravarthy
It is proved that the distribution of centroids of the RBF network ap proaches node density of the Self-Organizing Feature Map as a limit, suggesting a deeper connection between the fundamental learning paradigms, namely supervised and unsupervised learning.