Learning to Rank for Information Retrieval and Natural Language Processing
Synthesis lectures on human language technologiesPublished 22 April 2011Open access
Hang Li
Citations257
SJR quartileQ3
SJR score0.12
SNIP0.00
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
This lecture gives an introduction to the area including the fundamental problems, major approaches, theories, applications, and future work.
Keywords
Computer Science
Machine LearningSupport-Vector Networks
33,035 Citations1995Corinna Cortes, Vladimir Vapnik
High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
The Annals of StatisticsGreedy function approximation: A gradient boosting machine.
28,973 Citations2001Jerome H. Friedman
A general gradient descent boosting paradigm is developed for additive expansions based on any fitting criterion, and specific algorithms are presented for least-squares, least absolute deviation, and Huber-M loss functions for regression, and multiclass logistic likelihood for classification.
The PageRank Citation Ranking : Bringing Order to the Web
12,645 Citations1999Lawrence M. Page, Sergey Brin +2 more
This paper describes PageRank, a mathod for rating Web pages objectively and mechanically, effectively measuring the human interest and attention devoted to them, and shows how to efficiently compute PageRank for large numbers of pages.
Psychological ReviewThe perceptron: A probabilistic model for information storage and organization in the brain.
11,746 Citations1958Frank F. Rosenblatt
This article will be concerned primarily with the second and third questions, which are still subject to a vast amount of speculation, and where the few relevant facts currently supplied by neurophysiology have not yet been integrated into an acceptable theory.
Optimizing search engines using clickthrough data
3,898 Citations2002Thorsten Joachims
The goal of this paper is to develop a method that utilizes clickthrough data for training, namely the query-log of the search engine in connection with the log of links the users clicked on in the presented ranking.
A Short Introduction to Boosting
2,957 Citations1999Yoav Freund, Robert E. Schapire
This short overview paper introduces the boosting algorithm AdaBoost, and explains the underlying theory of boosting, including an explanation of why boosting often does not suffer from overfitting as well as boosting’s relationship to support-vector machines.
Learning to rank using gradient descent
2,759 Citations2005Chris Burges, Tal Shaked +5 more
RankNet is introduced, an implementation of these ideas using a neural network to model the underlying ranking function, and test results on toy data and on data from a commercial internet search engine are presented.
ACM SIGIR ForumA Language Modeling Approach to Information Retrieval
2,532 Citations2017Jay Ponte, W. Bruce Croft
It will be shown that probabilistic methods can be used to predict topic changes in the context of the task of new event detection and provide further proof of concept for the use of language models for retrieval tasks.
Learning to rank
1,917 Citations2007Zhe Cao, Tao Qin +3 more
It is proposed that learning to rank should adopt the listwise approach in which lists of objects are used as 'instances' in learning, and introduces two probability models, respectively referred to as permutation probability and top k probability, to define a listwise loss function for learning.
Rank aggregation methods for the Web
1,798 Citations2001Cynthia Dwork, Ravi Kumar +2 more
A set of techniques for the rank aggregation problem is developed and compared to that of well-known methods, to design rank aggregation techniques that can be used to combat spam in Web searches.
A language modeling approach to information retrieval
1,723 Citations1998Jay Ponte, W. Bruce Croft
This work proposes an approach to retrieval based on probabilistic language modeling and integrates document indexing and document retrieval into a single model, which significantly outperforms standard tf.idf weighting on two different collections and query sets.
Foundations and Trends® in Information RetrievalLearning to Rank for Information Retrieval
1,508 Citations2009Tie‐Yan Liu
ACM SIGIR ForumIR evaluation methods for retrieving highly relevant documents
1,466 Citations2017Kalervo Järvelin, Jaana Kekäläinen
The novel evaluation methods and the case demonstrate that non-dichotomous relevance assessments are applicable in IR experiments, may reveal interesting phenomena, and allow harder testing of IR methods.
ACM Transactions on Information SystemsA study of smoothing methods for language models applied to information retrieval
1,212 Citations2004ChengXiang Zhai, John Lafferty
Evaluation on five different databases and four types of queries indicates that the two-stage smoothing method with the proposed parameter estimation methods consistently gives retrieval performance that is close to or better than the best results achieved using a single smoothing methods and exhaustive parameter search on the test data.
Improving web search ranking by incorporating user behavior information
1,041 Citations2006Eugene Agichtein, Eric Brill +1 more
It is shown that incorporating user behavior data can significantly improve ordering of top results in real web search setting, improving the accuracy of a competitive web search ranking algorithms by as much as 31% relative to the original performance.
Elsevier eBooksCombating Web Spam with TrustRank
1,025 Citations2004Zoltán Gyöngyi, Héctor García-Molina +1 more
This paper proposes techniques to semi-automatically separate reputable, good pages from spam, and shows that they can effectively filter out spam from a significant fraction of the web, based on a good seed set of less than 200 sites.
IR evaluation methods for retrieving highly relevant documents
958 Citations2000Kalervo Järvelin, Jaana Kekäläinen
The novel evaluation methods and the case demonstrate that non-dichotomous relevance assessments are applicable in IR experiments, may reveal interesting phenomena, and allow harder testing of IR methods.
A Markov random field model for term dependencies
849 Citations2005Donald Metzler, W. Bruce Croft
A novel approach is developed to train the model that directly maximizes the mean average precision rather than maximizing the likelihood of the training data, and significant improvements are possible by modeling dependencies, especially on the larger web collections.
AdaRank
782 Citations2007Jun Xu, Hang Li
The proposed novel learning algorithm, referred to as AdaRank, repeatedly constructs 'weak rankers' on the basis of reweighted training data and finally linearly combines the weak rankers for making ranking predictions, which proves that the training process of AdaRank is exactly that of enhancing the performance measure used.
A support vector method for optimizing average precision
720 Citations2007Yisong Yue, Thomas Finley +2 more
This work presents a general SVM learning algorithm that efficiently finds a globally optimal solution to a straightforward relaxation of MAP, and shows its method to produce statistically significant improvements in MAP scores.
Listwise approach to learning to rank
694 Citations2008Fen Xia, Tie‐Yan Liu +3 more
A sufficient condition on consistency for ranking is given, which seems to be the first such result obtained in related research, and analysis on three loss functions: likelihood loss, cosine loss, and cross entropy loss are conducted.
Models for metasearch
681 Citations2001Javed Aslam, Mark Montague
The experimental results show that metasearch algorithms based on the Borda and Bayesian models usually outperform the best input system and are competitive with, and often outperform, existing metAsearch strategies.
Some Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval
634 Citations1994Stephen Robertson, Steve Walker
The 2-Poisson model for term frequencies is used to suggest ways of incorporating certain variables in probabilistic models for information retrieval, and substantial performance improvements are demonstrated.
Information RetrievalAdapting boosting for information retrieval measures
572 Citations2009Qiang Wu, Christopher J. C. Burges +2 more
This work presents a new ranking algorithm that combines the strengths of two previous methods: boosted tree classification, and LambdaRank, and shows how to find the optimal linear combination for any two rankers, and uses this method to solve the line search problem exactly during boosting.
Adapting ranking SVM to document retrieval
570 Citations2006Yunbo Cao, Jun Xu +4 more
Experimental results show that the modifications made in conventional Ranking SVM can outperform the conventional ranking SVM and other existing methods for document retrieval on two datasets and employ two methods to conduct optimization on the loss function: gradient descent and quadratic programming.
The MIT Press eBooksPranking with Ranking
550 Citations2002Koby Crammer, Yoram Singer
A simple and efficient online algorithm is described, its performance in the mistake bound model is analyzed, its correctness is proved, and it outperforms online algorithms for regression and classification applied to ranking.
Learning diverse rankings with multi-armed bandits
518 Citations2008Filip Radlinski, Robert Kleinberg +1 more
This work presents two online learning algorithms that directly learn a diverse ranking of documents based on users' clicking behavior and shows that these algorithms minimize abandonment, or alternatively, maximize the probability that a relevant document is found in the top k positions of a ranking.
Query chains
478 Citations2005Filip Radlinski, Thorsten Joachims
A novel approach for using clickthrough data to learn ranked retrieval functions for web search results by using query chains to generate new types of preference judgments from search engine logs, thus taking advantage of user intelligence in reformulating queries.
Information RetrievalLETOR: A benchmark collection for research on learning to rank for information retrieval
471 Citations2009Tao Qin, Tie‐Yan Liu +2 more
The details of the LETOR collection are described and it is shown how it can be used in different kinds of researches, and several state-of-the-art learning to rank algorithms on LETOR are compared.
Support vector learning for ordinal regression
458 Citations1999Ralf Herbrich
Experimental results indicate that the presented algorithm outperforms more naive approaches to ordinal regression such as support vector classification and support vector regression in the case of more than two ranks.
McRank: Learning to Rank Using Multiple Classification and Gradient Boosting
434 Citations2007Ping Li, Qiang Wu +1 more
This work considers the DCG criterion (discounted cumulative gain), a standard quality measure in information retrieval, and proposes using the Expected Relevance to convert class probabilities into ranking scores.
LETOR: Benchmark Dataset for Research on Learning to Rank for Information Retrieval
419 Citations2007Tie‐Yan Liu, Jun Xu +3 more
This paper has constructed a benchmark dataset referred to as LETOR, derived the LETOR data from the existing data sets widely used in IR, namely, OHSUMED and TREC data and provided the results of several state-ofthe-arts learning to rank algorithms on the data.
Scientific Repository (Petra Christian University)Gaussian Processes for Ordinal Regression
373 Citations2005Wei Chu, Zoubin Ghahramani
SoftRank
326 Citations2008Michael Taylor, John Guiver +2 more
This work presents a new family of training objectives that are derived from the rank distributions of documents, induced by smoothed scores, called SoftRank, and focuses on a smoothed approximation to Normalized Discounted Cumulative Gain (NDCG), called SoftNDCG.
Ranking with Large Margin Principle: Two Approaches
314 Citations2002Amnon Shashua, Anat Levin
Two main approaches to the problem of ranking k instances with the use of a "large margin" principle are introduced: the "fixed margin" policy in which the margin of the closest neighboring classes is being maximized and a direct generalization of SVM to ranking learning.
Journal of Artificial Intelligence ResearchLearning to Order Things
313 Citations1999W. W. Cohen, R. E. Schapire +1 more
An on-line algorithm for learning preference functions that is based on Freund and Schapire's "Hedge" algorithm is considered, and it is shown that the problem of finding the ordering that agrees best with a learned preference function is NP-complete.
New approaches to support vector ordinal regression
282 Citations2005Wei Chu, S. Sathiya Keerthi
Two new support vector approaches for ordinal regression are proposed, which optimize multiple thresholds to define parallel discriminant hyperplanes for the ordinal scales and guarantee that the thresholds are properly ordered at the optimal solution.
Feature selection for ranking
254 Citations2007Xiubo Geng, Tie‐Yan Liu +2 more
This paper proposes a new feature selection method that uses its value to rank the training instances, and defines the ranking accuracy in terms of a performance measure or a loss function as the importance of the feature.
Information RetrievalEfficient algorithms for ranking with SVMs
251 Citations2009Olivier Chapelle, S. Sathiya Keerthi
Evaluation on the Letor benchmark datasets after complete training using new methods based on primal Newton method to speed up RankSVM training and show that they are 5 orders of magnitude faster than SVMLight.
Lecture notes in computer scienceSubset Ranking Using Regression
245 Citations2006David Cossock, Tong Zhang
Borders are presented that relate the approximate optimization of DCG to the approximate minimization of certain regression errors and justify the use of convex learning formulations for solving the subset ranking problem.
An exploration of proximity measures in information retrieval
241 Citations2007Tao Tao, ChengXiang Zhai
This paper proposes and studies the effectiveness of five different proximity measures, each modeling proximity from a different perspective, and designs two heuristic constraints and uses them to guide us in incorporating the proposed proximity measures into an existing retrieval model.
FRank
222 Citations2007Ming-Feng Tsai, Tie‐Yan Liu +3 more
An algorithm named FRank is proposed based on a generalized additive model for the sake of minimizing the fedelity loss and learning an effective ranking function and the experimental results show that the proposed algorithm outperforms other learning-based ranking methods on both conventional IR problem and Web search.
Information RetrievalA general approximation framework for direct optimization of information retrieval measures
211 Citations2009Tao Qin, Tie‐Yan Liu +1 more
A general framework for direct optimization of IR measures, which enjoys several theoretical advantages, and experiments on benchmark datasets show that the algorithms deduced from the framework are very effective when compared to existing methods.
Discriminative Reranking for Machine Translation
207 Citations2004Libin Shen, Anoop Sarkar +1 more
This paper describes the application of discriminative reranking techniques to the problem of machine translation and introduces two novel perceptroninspired reranking algorithms that improve on the quality of machinetranslation over the baseline system based on evaluation using the BLEU metric.
A regression framework for learning ranking functions using relative relevance judgments
206 Citations2007Zhaohui Zheng, Keke Chen +2 more
This work focuses on developing a regression framework for learning ranking functions for improving relevance of search engines serving diverse streams of user queries, and proposes a novel optimization framework emphasizing the use of relative relevance judgments.
Information RetrievalGradient descent optimization of smoothed information retrieval metrics
205 Citations2009Olivier Chapelle, Mingrui Wu
This work proposes an algorithm which aims at directly optimizing popular measures such as the Normalized Discounted Cumulative Gain and the Average Precision, to minimize a smooth approximation of these measures with gradient descent.
A General Boosting Method and its Application to Learning Ranking Functions for Web Search
185 Citations2007Zhaohui Zheng, Hongyuan Zha +4 more
This work presents a general boosting method extending functional gradient boosting to optimize complex loss functions that are encountered in many machine learning problems, based on optimization of quadratic upper bounds of the loss functions.
Cranking: Combining Rankings Using Conditional Probability Models on Permutations
179 Citations2002Guy Lebanon, John Lafferty
A new approach to ensemble learning is introduced that takes ranking rather than classification as fundamental, leading to models on the symmetric group and its cosets, a generalization of the Mallows model on permutations to combine multiple input rankings.
BrowseRank
172 Citations2008Yuting Liu, Bin Gao +5 more
Experimental results show that BrowseRank indeed outperforms the baseline methods such as PageRank and TrustRank in several tasks.
Bayesian inference for Plackett-Luce ranking models
171 Citations2009John Guiver, Edward Snelson
An efficient Bayesian method for inferring the parameters of a Plackett-Luce ranking model is given and a number of advantages of the EP approach over the traditional maximum likelihood method are shown.
Query dependent ranking using K-nearest neighbor
169 Citations2008Xiubo Geng, Tie‐Yan Liu +4 more
This paper proposes a K-Nearest Neighbor (KNN) method for query-dependent ranking, and proves a theory which indicates that the approximations are accurate in terms of difference in loss of prediction, if the learning algorithm used is stable with respect to minor changes in training examples.
Predicting diverse subsets using structural SVMs
167 Citations2008Yisong Yue, Thorsten Joachims
This work formulate the learning problem of predicting diverse subsets and derive a training method based on structural SVMs that explicitly trains to diversify results.
Supervised rank aggregation
165 Citations2007Yuting Liu, Tie‐Yan Liu +3 more
Experimental results on meta-searches show that Supervised Rank Aggregation can significantly outperform existing unsupervised methods and it is proved that the optimization problem can be transformed into that of Semidefinite Programming and solve it efficiently.
Context-aware ranking in web search
161 Citations2010Biao Xiang, Daxin Jiang +4 more
The experimental results clearly show that the context-aware ranking approach improves the ranking of a commercial search engine which ignores context information and outperforms a baseline method which considers context information in ranking.
Learning to Rank by Optimizing NDCG Measure
160 Citations2009Hamed Valizadegan, Rong Jin +2 more
This work proposes a probabilistic framework that addresses the challenge of evaluating ranking algorithms using the expectation of NDCG over all the possible permutations of documents, and proposes an algorithm that outperforms state-of-the-art ranking algorithms on several benchmark data sets.
Towards recency ranking in web search
157 Citations2010Anlei Dong, Yi Chang +7 more
This paper proposes a retrieval system which automatically detects and responds to recency sensitive queries, and proposes several training methodologies important for training recencysensitive rankers.
Quality-biased ranking of web documents
155 Citations2011Michael Bendersky, W. Bruce Croft +1 more
This paper presents the quality-biased ranking method that promotes documents containing high-quality content, and penalizes low-quality documents, and consistently improves the retrieval performance of text-based and link-based retrieval methods that do not take into account the quality of the document content.
Active exploration for learning rankings from clickthrough data
153 Citations2007Filip Radlinski, Thorsten Joachims
This work develops a Bayesian approach for selecting rankings to present users so that interactions result in more informative training data and finds that active exploration substantially outperformassive observation and random exploration.
Ranking Measures and Loss Functions in Learning to Rank
149 Citations2009Wei Chen, Tie‐Yan Liu +3 more
The relationship between ranking measures and loss functions in learningto-rank methods, such as Ranking SVM, RankBoost, RankNet, and ListMLE are revealed, and it is proved that the essential loss is both an upper bound of the measure-based ranking errors, and a lower Bound of the loss functions of the aforementioned methods.
Ranking with ordered weighted pairwise classification
148 Citations2009Nicolas Usunier, David Buffoni +1 more
This work proposes to optimize a larger class of loss functions for ranking, based on an ordered weighted average (OWA) (Yager, 1988) of the classification losses, and shows that OWA aggregates of margin-based classification losses have good generalization properties.
A ranking approach to keyphrase extraction
140 Citations2009Xin Jiang, Yunhua Hu +1 more
Experimental results on three datasets show that Ranking SVM significantly outperforms the baseline methods of SVM and Naive Bayes, indicating that it is better to exploit learning to rank techniques in keyphrase extraction.
Decision tree and instance-based learning for label ranking
137 Citations2009Weiwei Cheng, Jens Hühn +1 more
New methods for label ranking are introduced that complement and improve upon existing approaches and are extensions of two methods that have been used extensively for classification and regression so far, namely instance-based learning and decision tree induction.
Information Processing & ManagementQuery-level loss functions for information retrieval
136 Citations2007Tao Qin, Xudong Zhang +4 more
A query-level loss function based on the cosine similarity between a ranking list and the corresponding ground truth is proposed and a coordinate descent algorithm is designed, referred to as RankCosine, which utilizes the proposed loss function to create a generalized additive ranking model.
High accuracy retrieval with multiple nested ranker
131 Citations2006Irina Matveeva, Chris Burges +3 more
This paper presents the multiple nested ranker approach that improves the accuracy at the top ranks by iteratively re-ranking the top scoring documents by using the RankNet learning algorithm to re-rank a subset of the results.
Directly optimizing evaluation measures in learning to rank
130 Citations2008Jun Xu, Tie‐Yan Liu +3 more
Experimental results show that the methods based on direct optimization of evaluation measures can always outperform conventional methods of Ranking SVM and RankBoost, however, no significant difference exists among the performances of the direct optimization methods themselves.
Structured learning for non-smooth ranking losses
123 Citations2008Soumen Chakrabarti, Rajiv Khanna +2 more
This paper proposes new, almost-linear-time algorithms to optimize for two other criteria widely used to evaluate search systems: MRR (mean reciprocal rank) and NDCG (normalized discounted cumulative gain) in the max-margin structured learning framework.
Global Ranking Using Continuous Conditional Random Fields
118 Citations2008Tao Qin, Tie‐Yan Liu +3 more
The paper shows how the Continuous CRF method can perform global ranking better than baselines and can naturally represent the content information of objects as well as the relation information between objects, necessary for global ranking.
Optimisation methods for ranking functions with multiple parameters
114 Citations2006Michael Taylor, Hugo Zaragoza +3 more
This work builds on recent advances in alternative differentiable pairwise cost functions, and shows that these techniques can be successfully applied to tuning the parameters of an existing family of IR scoring functions (BM25), in the sense that they cannot do better using sensible search heuristics that directly optimize the rank-based cost function NDCG.
Multi-task learning for boosting with application to web search ranking
113 Citations2010Olivier Chapelle, Pannagadatta K. Shivaswamy +4 more
A novel algorithm for multi-task learning with boosted decision trees that learns several different learning tasks with a joint model, explicitly addressing the specifics of each learning task with task-specific parameters and the commonalities between them through shared parameters.
BoltzRank
105 Citations2009Maksims Volkovs, Richard S. Zemel
This paper proposes a new listwise approach to learning to rank, which creates a conditional probability distribution over rankings assigned to documents for a given query, which permits gradient ascent optimization of the expected value of some performance measure.
Ranking with multiple hyperplanes
104 Citations2007Tao Qin, Xudong Zhang +4 more
This paper looks at an alternative approach to Ranking SVM, which it is called "Multiple Hyperplane Ranker" (MHR), and makes comparisons between the two approaches, which takes the divide-and-conquer strategy.
Magnitude-preserving ranking algorithms
104 Citations2007Corinna Cortes, Mehryar Mohri +1 more
This paper describes and analyzes several algorithms for ranking when one wishes not just to accurately predict pairwise ordering but also preserve the magnitude of the preferences or the difference between ratings, extending previously known stability results to non-bipartite ranking and magnitude of preference-preserving algorithms.
Learning to rank networked entities
103 Citations2006Alekh Agarwal, Soumen Chakrabarti +1 more
A framework for ranking networked entities based on Markov walks with parameterized conductance values associated with the network edges and a constrained maximum entropy network flow formulation whose dual can be solved efficiently using a cutting-plane approach and a quasi-Newton optimizer.
On the local optimality of LambdaRank
101 Citations2009Pınar Dönmez, Krysta M. Svore +1 more
It is shown that LambdaRank, which smoothly approximates the gradient of the target measure, can be adapted to work with four popular IR target evaluation measures using the same underlying gradient construction.
Linear discriminant model for information retrieval
100 Citations2005Jianfeng Gao, Haoliang Qi +2 more
Results show that in most test sets, LDM significantly outperforms the state-of-the-art language modeling approaches and the classical probabilistic retrieval model and it is more appropriate to train LDM using a measure of AP rather than likelihood if the IR system is graded on AP.
Learning to rank with partially-labeled data
97 Citations2008Kevin Duh, Katrin Kirchhoff
A framework for transductive learning of ranking functions is presented and it is shown that the answer is affirmative that unlabeled (test) data can be exploited to improve ranking performance.
arXiv (Cornell University)Direct Optimization of Ranking Measures
94 Citations2007Quoc V. Le, Alexander J. Smola
Key to the approach is that during training the ranking problem can be viewed as a linear assignment problem, which can be solved by the Hungarian Marriage algorithm, and a sort operation is sufficient, as the algorithm assigns a relevance score to every document, query pair.
Unsupervised rank aggregation with distance-based models
90 Citations2008Alexandre Klementiev, Dan Roth +1 more
This work proposes a mathematical and algorithmic framework for learning to aggregate (partial) rankings without supervision, instantiate the framework for the cases of combining permutations and combining top-k lists, and proposes a novel metric for the latter.
Active learning for ranking through expected loss optimization
83 Citations2010Bo Long, Olivier Chapelle +4 more
This paper derives a novel algorithm, expected discounted cumulative gain (DCG) loss optimization (ELO-DCG), to select most informative examples and investigates both query and document level active learning for raking and proposes a two-stage ELO- DCG algorithm which incorporate bothquery and document selection into active learning.
Learning to rank relational objects and its application to web search
80 Citations2008Tao Qin, Tie‐Yan Liu +4 more
Experimental results show that the proposed method outperforms the baseline methods for two ranking tasks (Pseudo Relevance Feedback and Topic Distillation) in web search, indicating that the suggested method can indeed make effective use of relation information and content information in ranking.
ACM SIGIR ForumLearning to rank for information retrieval (LR4IR 2007)
71 Citations2007Thorsten Joachims, Hang Li +2 more
The goal is to design and apply methods to automatically learn a function from training data, such that the function can sort objects according to their degrees of relevance, preference, or importance as defined in a specific application.
Document selection methodologies for efficient and effective learning-to-rank
66 Citations2009Javed A. Aslam, Evangelos Kanoulas +3 more
This paper employs a number of document selection methodologies, widely used in the context of evaluation--depth-k pooling, sampling, sampling (infAP, statAP), active-learning (MTC), and on-line heuristics (hedge), and investigates whether they can enable efficient and effective learning-to-rank.
A boosting algorithm for learning bipartite ranking functions with partially labeled data
64 Citations2008Massih Amini, Tuong Vinh Truong +1 more
The proposed approach is a semi-supervised inductive ranking algorithm which is able to infer an ordering on new examples that were not used for its training, and shows statistically significant improvements with respect to these ranking measures.
Learning to rank with SoftRank and Gaussian processes
62 Citations2008John Guiver, Edward Snelson
The SoftRank framework is extended to make use of the score uncertainties which are naturally provided by a Gaussian process (GP), which is a probabilistic non-linear regression model, which gives improved performance and efficiency.
Title extraction from bodies of HTML documents and its application to web page retrieval
61 Citations2005Yunhua Hu, Guomao Xin +5 more
Experimental results indicate that the use of both extracted titles and title fields is almost always better than theuse of title fields alone; theUse of extracted titles is particularly helpful in the task of named page finding.
Generating labels from clicks
59 Citations2009Rahul Agrawal, Alan Halverson +3 more
A novel way of transforming clicks into weighted, directed graphs inspired by eye-tracking studies is given and an objective function for finding cuts in these graphs that induce a good labeling is devised.
Ranking definitions with supervised learning methods
58 Citations2005Jun Xu, Yunbo Cao +2 more
Experimental results indicate that the use of SVM and Ranking SVM can significantly outperform the baseline methods of using heuristic rules or employing the conventional information retrieval method of Okapi, indicating that generic models for definition ranking can be constructed.
Generalization analysis of listwise learning-to-rank algorithms
56 Citations2009Yanyan Lan, Tie‐Yan Liu +2 more
A theorem is proved which gives a generalization bound of a listwise ranking algorithm, on the basis of Rademacher Average of the class of compound functions, which can naturally describe various listwise learning-to-rank algorithms.
On composition of a federated web search result page
53 Citations2011Ashok Kumar Ponnuswami, Kumaresh Pattabiraman +3 more
This paper presents a machine-learning framework for SERP composition in the presence of multiple relevant verticals and uses correlation of clicks as an offline metric and shows that click-preference target has a better correlation than human judgments based models.
arXiv (Cornell University)An efficient reduction of ranking to classification
52 Citations2007Nir Ailon, Mehryar Mohri
The reduction is randomized and guarantees a pairwise misranking regret bounded by that of the binary classifier, improving on a recent result of Balcan et al. (2007) and proving that randomization is necessary to achieve the reduction guarantees.
Information Processing & ManagementSemi-supervised document retrieval
46 Citations2008Ming Li, Hang Li +1 more
Experimental results indicate that SSRank consistently and almost always significantly outperforms the baseline methods, given the same amount of labeled data, because SSRank can effectively leverage the use of unlabeled data in learning.
Improving quality of training data for learning to rank using click-through data
45 Citations2010Jingfang Xu, Chuanliang Chen +3 more
This paper shows that the quality of training data labeled by humans has critical impact on the performance of learning to rank algorithms, and proposes a method to detect relevance judgment errors using click-through data accumulated at a search engine.
Information RetrievalKnowledge transfer for cross domain learning to rank
45 Citations2009Depin Chen, Yan Xiong +4 more
This paper aims at improving the learning of a ranking model in target domain by leveraging knowledge from the outdated or out-of-domain data by proposing two novel methods to conduct knowledge transfer at feature level and instance level.
IntervalRank
44 Citations2010Taesup Moon, Alex Smola +2 more
This paper shows how a combination of pointwise, pairwise, and list-wise approaches can lead to improved ranking performance and, moreover, how it can be implemented in log-linear time.
Multi-task learning for learning to rank in web search
41 Citations2009Jing Bai, Ke Zhou +6 more
A boosting framework for learning to rank in the multi- task learning context to learn non-parametric common structures adaptively from multiple tasks in a stage-wise way and demonstrates that multi-task learning methods bring significant relevance improvements over existing baseline methods.
Fast learning of document ranking functions with the committee perceptron
41 Citations2008Jonathan L. Elsas, Vitor R. Carvalho +1 more
This method performs comparably or better than two state-of-the-art rank learning algorithms, and also provides significant training time improvements over those methods, showing over a 45-fold reduction in training time compared to ranking SVM.
Automatic extraction of titles from general documents using machine learning
41 Citations2005Yunhua Hu, Hang Li +3 more
It turns out that the use of formatting information can lead to quite accurate extraction from general documents, and one can significantly improve search ranking results in do document retrieval by using the extracted titles.
Global ranking by exploiting user clicks
40 Citations2009Shihao Ji, Ke Zhou +6 more
This paper focuses on extracting relevance information from one source of user interactions, i.e., user click data, which records the sequence of documents being clicked and not clicked in the result set during a user search session, and adapt several sequential supervised learning algorithms to the global ranking problem.
Learning to rank only using training data from related domain
39 Citations2010Wei Gao, Peng Cai +2 more
This work proposes to exploit training data annotated for a related domain to learn to rank retrieved documents in the target domain, in which no labeled data is available, and presents a simple yet effective approach based on instance-weighting scheme.
…