Learning to rank using gradient descent

Published 1 January 2005

Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton

Citations2,759

TL;DR

RankNet is introduced, an implementation of these ideas using a neural network to model the underlying ranking function, and test results on toy data and on data from a commercial internet search engine are presented.

Abstract

We investigate using gradient descent methods for learning ranking functions; we propose a simple probabilistic cost function, and we introduce RankNet, an implementation of these ideas using a neural network to model the underlying ranking function. We present test results on toy data and on data from a commercial internet search engine. 1.

Keywords

Computer Science

Series in machine perception and artificial intelligenceSIGNATURE VERIFICATION USING A “SIAMESE” TIME DELAY NEURAL NETWORK

2,202 Citations1994Jane Bromley, JAMES W. BENTZ +6 more

International Journal of Pattern Recognition and Artificial IntelligenceSIGNATURE VERIFICATION USING A “SIAMESE” TIME DELAY NEURAL NETWORK

2,096 Citations1993Jane Bromley, JAMES W. BENTZ +6 more

An algorithm for verification of signatures written on a pen-input tablet based on a novel, artificial neural network called a "Siamese" neural network, which consists of two identical sub-networks joined at their outputs.

ACM SIGIR ForumIR evaluation methods for retrieving highly relevant documents

1,466 Citations2017Kalervo Järvelin, Jaana Kekäläinen

The novel evaluation methods and the case demonstrate that non-dichotomous relevance assessments are applicable in IR experiments, may reveal interesting phenomena, and allow harder testing of IR methods.

The Annals of StatisticsClassification by pairwise coupling

1,291 Citations1998Trevor Hastie, Robert Tibshirani

A strategy for polychotomous classification that involves estimating class probabilities for each pair of classes, and then coupling the estimates together is discussed, similar to the Bradley-Terry method for paired comparisons.

View PDF

Journal of Mathematical Analysis and ApplicationsSome results on Tchebycheffian spline functions

1,255 Citations1971George Kimeldorf, Grace Wahba

Boosting Algorithms as Gradient Descent

712 Citations1999Llew Mason, Jonathan Baxter +2 more

Following previous theoretical results bounding the generalization performance of convex combinations of classifiers in terms of general cost functions of the margin, a new algorithm (DOOM II) is presented for performing a gradient descent optimization of such cost functions.

The MIT Press eBooksPranking with Ranking

550 Citations2002Koby Crammer, Yoram Singer

A simple and efficient online algorithm is described, its performance in the mistake bound model is analyzed, its correctness is proved, and it outperforms online algorithms for regression and classification applied to ranking.

International Conference on Machine LearningSimplified support vector decision rules

429 Citations1996Christopher J. C. Burges

The results show that the method can decrease the computational complexity of the decision rule by a factor of ten with no loss in generalization perfor mance making the SVM test speed com petitive with that of other methods.

Log-Linear Models for Label Ranking

163 Citations2003Ofer Dekel, Yoram Singer +1 more

This work presents a general boosting-based learning algorithm for the label ranking problem and proves a lower bound on the progress of each boosting iteration.

Neural Information Processing SystemsSupervised Learning of Probability Distributions by Neural Networks

156 Citations1987Eric B. Baum, Frank Wilczek

The back propagation algorithm for supervised learning can be generalized, put on a satisfactory conceptual footing, and very likely made more efficient by defining the values of the output and input neurons as probabilities and varying the synaptic weights in the gradient direction of the log likelihood, rather than the 'error'.