Supervised Dictionary Learning
University of Minnesota Digital Conservancy (University of Minnesota)Published 18 September 2008Open access
Julien Mairal, Francis Bach, Jean Ponce, Guillermo Sapiro, Andrew Zisserman
Citations767
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
It is now well established that sparse signal models are well suited to restoration tasks and caneffectivelybelearnedfromaudio, image, and video data. Recentresearch has been aimed at learning discriminative sparse models instead of purely reconstructiveones. Thispaperproposesanewstepinthatdirection, with a novel sparserepresentationforsignalsbelongingto different classes in terms of a shared dictionaryandmultiplediscriminativeclassmodels. The linear variant of the proposed model admits a simple probabilistic interpretation, while its most general variant admits an interpretation in terms of kernels. An optimization framework for learning all the components of the proposed model is presented, along with experimentalresultson standard handwritten digit and texture classification tasks.
Keywords
Computer Science
Proceedings of the IEEEGradient-based learning applied to document recognition
58,219 Citations1998Yann LeCun, Léon Bottou +2 more
This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task, and Convolutional neural networks are shown to outperform all other techniques.
IEEE Transactions on Signal Processing$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
9,506 Citations2006Michal Aharon, Michael Elad +1 more
A novel algorithm for adapting dictionaries in order to achieve sparse signal representations, the K-SVD algorithm, an iterative method that alternates between sparse coding of the examples based on the current dictionary and a process of updating the dictionary atoms to better fit the data.
The Annals of StatisticsLeast angle regression
9,493 Citations2004Bradley Efron, Trevor Hastie +2 more
IEEE Transactions on Pattern Analysis and Machine IntelligenceRobust Face Recognition via Sparse Representation
9,444 Citations2009John Wright, A. Yang +3 more
This work considers the problem of automatically recognizing human faces from frontal views with varying expression and illumination, as well as occlusion and disguise, and proposes a general classification algorithm for (image-based) object recognition based on a sparse representation computed by C1-minimization.
IEEE Transactions on Image ProcessingImage Denoising Via Sparse and Redundant Representations Over Learned Dictionaries
5,385 Citations2006Michael Elad, Michal Aharon
This work addresses the image denoising problem, where zero-mean white and homogeneous Gaussian additive noise is to be removed from a given image, and uses the K-SVD algorithm to obtain a dictionary that describes the image content effectively.
Vision ResearchSparse coding with an overcomplete basis set: A strategy employed by V1?
3,720 Citations1997Bruno A. Olshausen, David J. Field
These deviations from linearity provide a potential explanation for the weak forms of non-linearity observed in the response properties of cortical simple cells, and they further make predictions about the expected interactions among units in response to naturalistic stimuli.
Self-taught learning
1,533 Citations2007Rajat Raina, Alexis Battle +3 more
An approach to self-taught learning that uses sparse coding to construct higher-level features using the unlabeled data to form a succinct input representation and significantly improve classification performance.
arXiv (Cornell University)Supervised Topic Models
1,315 Citations2010David M. Blei, Jon McAuliffe
The supervised latent Dirichlet allocation (sLDA) model, a statistical model of labelled documents, is introduced, which derives a maximum-likelihood procedure for parameter estimation, which relies on variational approximations to handle intractable posterior expectations.
Object categorization by learned universal visual dictionary
840 Citations2005John Winn, Antonio Criminisi +1 more
An optimally compact visual dictionary is learned by pair-wise merging of visual words from an initially large dictionary, and a novel statistical measure of discrimination is proposed which is optimized by each merge operation.
The MIT Press eBooksSparse Representation for Signal Classification
616 Citations2007Ke Huang, Selin Aviyente
The proposed approach combines the discrimination power of the discriminative methods with the reconstruction property and the sparsity of the sparse representation that enables one to deal with signal corruptions: noise, missing data and outliers.
Principled Hybrids of Generative and Discriminative Models
339 Citations2006Julia Lasserre, C.M. Bishop +1 more
A new perspective is adopted which says that there is only one correct way to train a given model, and that a ‘discriminatively trained’ generative model is fundamentally a new model and opens door to very general ways of interpolating between generative and discriminative extremes through alternative choices of prior.
Semi-supervised learning of compact document representations with deep networks
223 Citations2008Marc ' Aurelio Ranzato, Martin Szummer
An algorithm to learn text document representations based on semi-supervised autoencoders that are stacked to form a deep network that can be trained efficiently on partially labeled corpora, producing very compact representations of documents, while retaining as much class information and joint word statistics as possible.
Sparse Representations for Image Classification: Learning Discriminative and Reconstructive Non-Parametric Dictionaries
134 Citations2008Fernando Rodríguez, Guillermo Sapiro
A framework for learning optimal dictionaries for simultaneous sparse signal representation and robust class classification is introduced, addressing for the first time the explicit incorporation of both reconstruction and discrimination terms in the non-parametric dictionary learning and sparse coding energy.