Manifold elastic net: a unified framework for sparse dimension reduction

By using a series of equivalent transformations, it is shown MEN is equivalent to the lasso penalized least square problem and thus LARS is adopted to obtain the optimal sparse solution of MEN.

Abstract

It is difficult to find the optimal sparse solution of a manifold learning\nbased dimensionality reduction algorithm. The lasso or the elastic net\npenalized manifold learning based dimensionality reduction is not directly a\nlasso penalized least square problem and thus the least angle regression (LARS)\n(Efron et al. \\cite{LARS}), one of the most popular algorithms in sparse\nlearning, cannot be applied. Therefore, most current approaches take indirect\nways or have strict settings, which can be inconvenient for applications. In\nthis paper, we proposed the manifold elastic net or MEN for short. MEN\nincorporates the merits of both the manifold learning based dimensionality\nreduction and the sparse learning based dimensionality reduction. By using a\nseries of equivalent transformations, we show MEN is equivalent to the lasso\npenalized least square problem and thus LARS is adopted to obtain the optimal\nsparse solution of MEN. In particular, MEN has the following advantages for\nsubsequent classification: 1) the local geometry of samples is well preserved\nfor low dimensional data representation, 2) both the margin maximization and\nthe classification error minimization are considered for sparse projection\ncalculation, 3) the projection matrix of MEN improves the parsimony in\ncomputation, 4) the elastic net penalty reduces the over-fitting problem, and\n5) the projection matrix of MEN can be interpreted psychologically and\nphysiologically. Experimental evidence on face recognition over various popular\ndatasets suggests that MEN is superior to top level dimensionality reduction\nalgorithms.\n