A unified framework for semi-supervised dimensionality reduction
Pattern RecognitionPublished 15 January 2008
Yangqiu Song, Feiping Nie, Changshui Zhang, Shiming Xiang
Citations174
SJR quartileQ1
SJR score2.06
SNIP2.67
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper proposes a semi-supervised dimensionality reduction framework, which can efficiently handle the unlabeled data and can significantly improve the accuracy rates of the corresponding supervised and unsupervised approaches.
Abstract
In practice, many applications require a dimensionality reduction method to deal with the partially labeled problem. In this paper, we propose a semi-supervised dimensionality reduction framework, which can efficiently handle the unlabeled data. Under the framework, several classical methods, such as principal component analysis (PCA), linear discriminant analysis (LDA), maximum margin criterion (MMC), locality preserving projections (LPP) and their corresponding kernel versions can be seen as special cases. For high-dimensional data, we can give a low-dimensional embedding result for both discriminating multi-class sub-manifolds and preserving local manifold structure. Experiments show that our algorithms can significantly improve the accuracy rates of the corresponding supervised and unsupervised approaches. © 2008 Elsevier Ltd. All rights reserved.
Keywords
Computer ScienceSocial Sciences
IEEE Transactions on Pattern Analysis and Machine IntelligenceEigenfaces vs. Fisherfaces: recognition using class specific linear projection
11,753 Citations1997Peter N. Belhumeur, João P. Hespanha +1 more
Elsevier eBooksIntroduction to Statistical Pattern Recognition
11,121 Citations1990Keinosuke Fukunaga
A system to determine the sex of fish from their length, as accurately as possible, using a set of labelled data, together with the class label (male or female), as a training set.
Face recognition using eigenfaces
5,211 Citations2002Matthew Turk, Alex Pentland
An approach to the detection and identification of human faces is presented, and a working, near-real-time face recognition system which tracks a subject's head and then recognizes the person by comparing characteristics of the face to those of known individuals is described.
IEEE Transactions on Pattern Analysis and Machine IntelligenceFrom few to many: illumination cone models for face recognition under variable lighting and pose
4,915 Citations2001Athinodoros S. Georghiades, Peter N. Belhumeur +1 more
A generative appearance-based method for recognizing human faces under variation in lighting and viewpoint that exploits the fact that the set of images of an object in fixed pose but under all possible illumination conditions, is a convex cone in the space of images.
The MIT Press eBooksLaplacian Eigenmaps and Spectral Techniques for Embedding and Clustering
4,535 Citations2002Mikhail Belkin, Partha Niyogi
The algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality preserving properties and a natural connection to clustering.
IEEE Transactions on Pattern Analysis and Machine IntelligenceFace recognition using Laplacianfaces
3,276 Citations2005Xiaofei He, Shuicheng Yan +3 more
Experimental results suggest that the proposed Laplacianface approach provides a better representation and achieves lower error rates in face recognition.
Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples
3,266 Citations2006Mikhail Belkin, Partha Niyogi +1 more
A semi-supervised framework that incorporates labeled and unlabeled data in a general-purpose learner is proposed and properties of reproducing kernel Hilbert spaces are used to prove new Representer theorems that provide theoretical basis for the algorithms.
Parameterisation of a stochastic model for human face identification
2,586 Citations2002Ferdinando Samaria, Augustin Harter
This paper presents a set of experimental results in which various HMM parameterisations are analysed and shows that stochastic modelling can be used successfully to encode feature information.
Journal of Machine Learning ResearchManifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples
2,130 Citations2006BelkinMikhail, NiyogiPartha +1 more
Neural ComputationGeneralized Discriminant Analysis Using a Kernel Approach
1,679 Citations2000G. Baudat, F. Anouar
A new method that is close to the support vector machines insofar as the GDA method provides a mapping of the input vectors into high-dimensional feature space to deal with nonlinear discriminant analysis using kernel function operator.
Lecture notes in computer scienceA Generalized Representer Theorem
1,587 Citations2001Bernhard Schölkopf, Ralf Herbrich +1 more
The result shows that a wide range of problems have optimal solutions that live in the finite dimensional span of the training examples mapped into feature space, thus enabling us to carry out kernel algorithms independent of the (potentially infinite) dimensionality of the feature space.
The Annals of StatisticsPenalized Discriminant Analysis
864 Citations1995Trevor Hastie, Andreas Buja +1 more
A penalized version of Fisher's linear discriminant analysis is described, designed for situations in which there are many highly correlated predictors, such as those obtained by discretizing a function, or the grey-scale values of the pixels in a series of images.
IEEE Transactions on Neural NetworksEfficient and Robust Feature Extraction by Maximum Margin Criterion
845 Citations2006Haizhou Li, Tao Jiang +1 more
This paper proposes some new feature extractors based on maximum margin criterion (MMC) and establishes a new linear feature extractor that does not suffer from the small sample size problem, which is known to cause serious stability problems for LDA.
IEEE Transactions on Pattern Analysis and Machine IntelligenceKPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition
831 Citations2005Jian Yang, Alejandro F. Frangi +3 more
A two-phase KFD framework is developed, i.e., kernel principal component analysis (KPCA) plus Fisher linear discriminant analysis (LDA), which provides novel insights into the nature of KFD.
Semi-supervised Discriminant Analysis
760 Citations2007Deng Cai, Xiaofei He +1 more
This paper proposes a novel method, called Semi- supervised Discriminant Analysis (SDA), which makes use of both labeled and unlabeled samples to learn a discriminant function which is as smooth as possible on the data manifold.
Characterising Virtual Eigensignatures for General Purpose Face Recognition
619 Citations1998Daniel B. Graham, N.M. Allinson
An eigenspace manifold for the representation and recognition of pose-varying faces is described and a framework is proposed which can be used for both familiar and unfamiliar face recognition.
Semi-Supervised Dimensionality Reduction
315 Citations2007Daoqiang Zhang, Zhi‐Hua Zhou +1 more
This paper proposes the SSDR algorithm, which can preserve the intrinsic structure of the unlabeled data as well as both the must-link and cannot-link constraints defined on the labeled examples in the projected low-dimensional space.
Least squares linear discriminant analysis
277 Citations2007Jieping Ye
The equivalence relationship between the proposed least squares formulation and LDA for multi-class classifications is rigorously established under a mild condition, which is shown empirically to hold in many applications involving high-dimensional data.
OpenGrey (Institut de l'Information Scientifique et Technique)Nonlinear Discriminant Analysis Using Kernel Functions
231 Citations1999Volker Röth, Volker Steinhage
The presented algorithm allows a simple formulation of the EM-algorithm in terms of kernel functions which leads to a unique concept for unsupervised mixture analysis, supervised discriminant analysis and semi-supervised discriminantAnalysis with partially unlabelled observations in feature spaces.
Invariant Feature Extraction and Classification in Kernel Spaces
174 Citations1999Sebastian Mika, Gunnar Rätsch +4 more
Employing a unified framework in terms of a nonlinear variant of the Rayleigh coefficient, this work proposes non-linear generalizations of Fisher's discriminant and oriented PCA using Support Vector kernel functions.
Semi-supervised nonlinear dimensionality reduction
124 Citations2006Xin Yang, Haoying Fu +2 more
The sensitivity analysis of the algorithms shows that prior information will improve stability of the solution, and some insight is given on what kind of prior information best improves the solution.
Neural NetworksNonlinear Fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm
112 Citations2002S.A. Billings, Kian L. Lee
Two simple classification techniques will be introduced and tested on a number of real and artificial data sets and show that the new classification technique can often perform favourably compared with other state of the art classification techniques.
Discriminant Analysis: A Least Squares Approximation View
11 Citations2006Peng Zhang, Jing Peng +1 more
This paper derives a new approach to discriminant analysis based on statistical learning theory called discriminant learning analysis (DLA), which is wellposed and behaves well in the SSS situation.
Lecture notes in computer scienceDimensionality Reduction for Semi-supervised Face Recognition
10 Citations2005Weiwei Du, Kohei Inoue +1 more
A dimensionality reduction technique is presented for semi-supervised face recognition where image data are mapped into a low dimensional space with a spectral method and it is shown with experiments that the spectral mapping method outperforms the regularized regression.
Lecture notes in computer scienceAn Efficient Method to Estimate Labelled Sample Size for Transductive LDA(QDA/MDA) Based on Bayes Risk
5 Citations2004Han Liu, Xiao‐bing Yuan +2 more
A method to estimate the number of labels required in a dataset for semi-supervised linear discriminant classifiers (Transductive LDA) to reach an desired accuracy and extends naturally to handle two difficult problems: learning from gaussian distributions with different covariances, and learning for multiple classes.