Beyond data

Published 27 October 2013

Xavier Amatriain

Citations15

TL;DR

The different approaches Netflix follows to deal with large streams of user data in order to extract information for personalizing its service are discussed, and some of the machine learning models used and their application in the service are described.

Abstract

Since the Netflix $1 million Prize, announced in 2006, Netflix has been known for having personalization at the core of our product. Our current product offering is nowadays focused around instant video streaming, and our data is now many orders of magnitude larger. Not only do we have many more users in many more countries, but we also receive many more streams of data. Besides the ratings, we now also use information such as what our members play, browse, or search.

Keywords

Computer Science

ACM Transactions on Information SystemsEvaluating collaborative filtering recommender systems

5,762 Citations2004Jonathan L. Herlocker, Joseph A. Konstan +2 more

The key decisions in evaluating collaborative filtering recommender systems are reviewed: the user tasks being evaluated, the types of analysis and datasets being used, the ways in which prediction quality is measured, the evaluation of prediction attributes other than quality, and the user-based evaluation of the system as a whole.

Factorization meets the neighborhood

3,876 Citations2008Yehuda Koren

The factor and neighborhood models can now be smoothly merged, thereby building a more accurate combined model and a new evaluation metric is suggested, which highlights the differences among methods, based on their performance at a top-K recommendation task.

Restricted Boltzmann machines for collaborative filtering

1,882 Citations2007Ruslan Salakhutdinov, Andriy Mnih +1 more

This paper shows how a class of two-layer undirected graphical models, called Restricted Boltzmann Machines (RBM's), can be used to model tabular data, such as user's ratings of movies, and demonstrates that RBM's can be successfully applied to the Netflix data set.

Collaborative filtering with temporal dynamics

1,170 Citations2009Yehuda Koren

Two leading collaborative filtering recommendation approaches are revamp and a more sensitive approach is required, which can make better distinctions between transient effects and long term patterns.

ACM SIGKDD Explorations NewsletterLessons from the Netflix prize challenge

723 Citations2007Robert M. Bell, Yehuda Koren

This article outlines the overall strategy and summarizes a few key innovations of the team that won the first Netflix progress prize.

Trustworthy online controlled experiments

224 Citations2012Ron Kohavi, Alex Deng +4 more

The topics covered include: the OEC (Overall Evaluation Criterion), click tracking, effect trends, experiment length and power, and carryover effects, which should help readers increase the trustworthiness of the results coming out of controlled experiments.

Lecture notes in computer scienceI Like It... I Like It Not: Evaluating User Ratings Noise in Recommender Systems

195 Citations2009Xavier Amatriain, Josep M. Pujol +1 more

This paper presents a user study aimed at quantifying the noise in user ratings that is due to inconsistencies, and analyzes how factors such as item sorting and time of rating affect this noise.

ACM SIGKDD Explorations NewsletterMining large streams of user data for personalized recommendations

128 Citations2013Xavier Amatriain

An up-to-date overview of the use of data mining approaches for personalization and recommendation at Netflix and the most promising current research avenues and unsolved problems that deserve attention are pinpointed.