Maximum-Margin Matrix Factorization

Published 1 December 2004

Nathan Srebro, Jason D. M. Rennie, Tommi Jaakkola

Citations952

TL;DR

A novel approach to collaborative prediction is presented, using low-norm instead of low-rank factorizations, inspired by, and has strong connections to, large-margin linear discrimination.

Abstract

We present a novel approach to collaborative prediction, using low-norm instead of low-rank factorizations. The approach is inspired by, and has strong connections to, large-margin linear discrimination. We show how to learn low-norm factorizations by solving a semi-definite program, and discuss generalization error bounds for them. 1

Keywords

Computer Science

NatureLearning the parts of objects by non-negative matrix factorization

14,055 Citations1999Daniel D. Lee, H. Sebastian Seung

An algorithm for non-negative matrix factorization is demonstrated that is able to learn parts of faces and semantic features of text and is in contrast to other methods that learn holistic, not parts-based, representations.

Machine LearningUnsupervised Learning by Probabilistic Latent Semantic Analysis

2,443 Citations2001Thomas Hofmann

This paper proposes to make use of a temperature controlled version of the Expectation Maximization algorithm for model fitting, which has shown excellent performance in practice, and results in a more principled approach with a solid foundation in statistical inference.

View PDF

ACM Transactions on Information SystemsLatent semantic models for collaborative filtering

1,385 Citations2004Thomas Hofmann

A new family of model-based algorithms designed for collaborative filtering rely on a statistical modelling technique that introduces latent class variables in a mixture model setting to discover user communities and prototypical interest profiles.

A rank minimization heuristic with application to minimum order system approximation

994 Citations2001Maryam Fazel, H. Hindi +1 more

It is shown that the heuristic to replace the (nonconvex) rank objective with the sum of the singular values of the matrix, which is the dual of the spectral norm, can be reduced to a semidefinite program, hence efficiently solved.

International Conference on Machine LearningWeighted low-rank approximations

702 Citations2003Nathan Srebro, Tommi Jaakkola

This work provides a simple and efficient algorithm for solving weighted low-rank approximation problems, which, unlike their unweighted version, do not admit a closed-form solution in general.

Optimization methods & softwareCSDP, A C library for semidefinite programming

503 Citations1999Brian Borchers

CSDP is a library of routines that implements a predictor corrector variant of the semidefinite programming algorithm of Helmberg, Rendl, Vanderbei, and Wolkowicz that includes support for linear inequality constraints in addition to linear equality constraints.

A Generalization of Principal Components Analysis to the Exponential Family

387 Citations2001Michael Collins, Sanjoy Dasgupta +1 more

Ranking with Large Margin Principle: Two Approaches

314 Citations2002Amnon Shashua, Anat Levin

Two main approaches to the problem of ranking k instances with the use of a "large margin" principle are introduced: the "fixed margin" policy in which the margin of the closest neighboring classes is being maximized and a direct generalization of SVM to ranking learning.

Modeling User Rating Profiles For Collaborative Filtering

267 Citations2003Benjamin M. Marlin

A generative latent variable model for rating-based collaborative filtering called the User Rating Profile model (URP), which represents each user as a mixture of user attitudes, and the mixing proportions are distributed according to a Dirichlet random variable.

DSpace@MIT (Massachusetts Institute of Technology)Learning with matrix factorizations

218 Citations2004Nathan Srebro, Tommi Jaakkola

This thesis addresses several issues related to learning with matrix factorizations, study the asymptotic behavior and generalization ability of existing methods, suggest new optimization methods, and present a novel maximum-margin high-dimensional matrix factorization formulation.

View PDF