Regularized Distance Metric Learning:Theory and Algorithm
Published 7 December 2009
Rong Jin, Shijun Wang, Yang Zhou
Citations181
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is shown that with appropriate constraints, the generalization error of regularized distance metric learning could be independent from the dimensionality, making it suitable for handling high dimensional data.
Abstract
In this paper, we examine the generalization error of regularized distance metric learning. We show that with appropriate constraints, the generalization error of regularized distance metric learning could be independent from the dimensionality, making it suitable for handling high dimensional data. In addition, we present an efficient online learning algorithm for regularized distance metric learning. Our empirical studies with data classification and face recognition show that the proposed algorithm is (i) effective for distance metric learning when compared to the state-of-the-art methods, and (ii) efficient and robust for high dimensional data. 1
Keywords
Computer ScienceEngineering
ScienceA Global Geometric Framework for Nonlinear Dimensionality Reduction
13,740 Citations2000Joshua B. Tenenbaum, Vin de Silva +1 more
An approach to solving dimensionality reduction problems that uses easily measured local metric information to learn the underlying global geometry of a data set and efficiently computes a globally optimal solution, and is guaranteed to converge asymptotically to the true structure.
Journal of Machine Learning ResearchDistance Metric Learning for Large Margin Nearest Neighbor Classification
4,253 Citations2009Kilian Q. Weinberger, Lawrence K. Saul
This paper shows how to learn a Mahalanobis distance metric for kNN classification from labeled examples in a globally integrated manner and finds that metrics trained in this way lead to significant improvements in kNN Classification.
Cambridge University Press eBooksPrediction, Learning, and Games
3,258 Citations2006Nicolò Cesa‐Bianchi, Gábor Lugosi
This chapter discusses prediction with expert advice, efficient forecasters for large classes of experts, and randomized prediction for specific losses.
Distance Metric Learning with Application to Clustering with Side-Information
2,710 Citations2002Eric P. Xing, Michael I. Jordan +2 more
This paper presents an algorithm that, given examples of similar (and, if desired, dissimilar) pairs of points in �”n, learns a distance metric over ℝn that respects these relationships.
An Introduction to the Conjugate Gradient Method Without the Agonizing Pain
2,211 Citations1994Jonathan Richard Shewchuk
Information-theoretic metric learning
2,040 Citations2007Jason V. Davis, Brian Kulis +3 more
An information-theoretic approach to learning a Mahalanobis distance function that can handle a wide variety of constraints and can optionally incorporate a prior on the distance function and derive regret bounds for the resulting algorithm.
Distance Metric Learning for Large Margin Nearest Neighbor Classification
1,707 Citations2005Kilian Q. Weinberger, John Blitzer +1 more
Choice Reviews OnlineNonparametric statistics for non-statisticians: a step-by-step approach
1,030 Citations2010
Applied Physics Letters10.1162/153244302760200704
696 Citations2000
It is shown how to use notions of stability for learning algorithms to derive generalization error bounds based on the empirical error and the leave-one-out error to apply to SVM for regression and classification.
Metric Learning by Collapsing Classes
663 Citations2005Amir Globerson, Sam T. Roweis
An algorithm for learning a quadratic Gaussian metric (Mahalanobis distance) for use in classification tasks and discusses how the learned metric may be used to obtain a compact low dimensional feature representation of the original input space, allowing more efficient classification with very little reduction in performance.
Applied Physics Letters10.1162/153244304322972667
353 Citations2000
Locally linear embedding (LLE) is described, an unsupervised learning algorithm that computes low dimensional, neighborhood preserving embeddings of high dimensional data that relates to previous and ongoing work in nonlinear dimensionality reduction.
Learning Distance Metrics with Contextual Constraints for Image Retrieval
316 Citations2006Steven C. H. Hoi, Wei Liu +2 more
Experimental results show that the proposed algorithms for Discriminative Component Analysis and Kernel DCA are effective and promising in learning good quality distance metrics for image retrieval.
Online and batch learning of pseudo-metrics
301 Citations2004Shai Shalev‐Shwartz, Yoram Singer +1 more
An online algorithm for supervised learning of pseudo-metrics that is based on successive projections onto the positive semi-definite cone and onto half-space constraints imposed by the examples is described and analyzed.
Lecture notes in computer scienceAdjustment Learning and Relevant Component Analysis
231 Citations2002Noam Shental, Tomer Hertz +2 more
A new learning approach for image retrieval is proposed, which is called adjustment learning, and this scheme uses the information in chunklets to reduce irrelevant variability in the data while amplifying relevant variability.
Semi-supervised distance metric learning for Collaborative Image Retrieval
162 Citations2008Steven C. H. Hoi, Wei Liu +1 more
A novel semi-supervised distance metric learning technique, called ldquoLaplacian Regularized Metric Learningrdquo (LRML), for learning robust distance metrics for CIR, which shows that reliable metrics can be learned from real log data even they may be noisy and limited at the beginning stage of a CIR system.
An efficient algorithm for local distance metric learning
146 Citations2006Liu Yang, Rong Jin +2 more
An efficient algorithm that employs eigenvector analysis, and bound optimization to learn the LDM from training data in a probabilistic framework is presented and it is demonstrated that LDM achieves significant improvements in both classification and retrieval accuracy compared to global distance learning and kernel-based KNN.
Distance metric learning from uncertain side information with application to automated photo tagging
92 Citations2009Lei Wu, Steven C. H. Hoi +3 more
A novel probabilistic distance metric learning scheme is proposed that automatically derives constraints from the uncertain side information, and efficiently learns a distance metric from the derived constraints.
Multimedia SystemsCollaborative image retrieval via regularized metric learning
56 Citations2006Luo Si, Rong Jin +2 more
This paper presents a novel metric learning approach, named “regularized metric learning,” for collaborative image retrieval, which learns a distance metric by exploring the correlation between low-level image features and the log data of users' relevance judgments.
Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005.Kernel relevant component analysis for distance metric learning
45 Citations2006Ivor W. Tsang, Pak-Ming Cheung +1 more
It is shown that RCA can also be kernelized, which then results in significant improvements when nonlinearities are needed, and becomes applicable to distance metric learning for structured objects that have no natural vectorial representation.