Correcting Sample Selection Bias by Unlabeled Data
The MIT Press eBooksPublished 7 September 2007
Jiayuan Huang, Alexander J. Smola, Arthur Gretton, Karsten Borgwardt, Bernhard Schölkopf
Citations1,535
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A nonparametric method which directly produces resampling weights without distribution estimation is presented, which works by matching distributions between training and testing sets in feature space.
Abstract
We consider the scenario where training and test data are drawn from different distributions, commonly referred to as sample selection bias.Most algorithms for this setting try to first recover sampling distributions and then make appropriate corrections based on the distribution estimate.We present a nonparametric method which directly produces resampling weights without distribution estimation.Our method works by matching distributions between training and testing sets in feature space.Experimental results demonstrate that our method works well in practice.
Keywords
Computer Science
Springer series in statisticsProbability Inequalities for sums of Bounded Random Variables
6,947 Citations1994Wassily Hoeffding
Neural ComputationEstimating the Support of a High-Dimensional Distribution
5,946 Citations2001Bernhard Schölkopf, John Platt +3 more
The algorithm is a natural extension of the support vector algorithm to the case of unlabeled data by carrying out sequential optimization over pairs of input patterns and providing a theoretical analysis of the statistical performance of the algorithm.
Statistical Inference
5,368 Citations1993George Casella, Roger L. Berger
Cancer CellGene expression correlates of clinical prostate cancer behavior
2,537 Citations2002Dinesh Singh, Phillip G. Febbo +13 more
The results support the notion that the clinical behavior of prostate cancer is linked to underlying gene expression differences that are detectable at the time of diagnosis.
MPG.PuRe (Max Planck Society)Large Margin Methods for Structured and Interdependent Output Variables
1,952 Citations2005Ioannis Tsochantaridis, Thorsten Joachims +2 more
This paper proposes to appropriately generalize the well-known notion of a separation margin and derive a corresponding maximum-margin formulation and presents a cutting plane algorithm that solves the optimization problem in polynomial time for a large class of problems.
The MIT Press eBooksA Kernel Method for the Two-Sample-Problem
1,899 Citations2007Arthur Gretton, Karsten Borgwardt +3 more
This work proposes two statistical tests to determine if two samples are from different distributions, and applies this approach to a variety of problems, including attribute matching for databases using the Hungarian marriage method, where the test performs strongly.
Journal of Statistical Planning and InferenceImproving predictive inference under covariate shift by weighting the log-likelihood function
1,681 Citations2000Hidetoshi Shimodaira
A class of predictive densities is derived by weighting the observed samples in maximizing the log-likelihood function, effective in cases such as sample surveys or design of experiments, where the observed covariate follows a different distribution than that in the whole population.
The MIT Press eBooksCorrecting Sample Selection Bias by Unlabeled Data
1,535 Citations2007Jiayuan Huang, Alexander J. Smola +3 more
A nonparametric method which directly produces resampling weights without distribution estimation is presented, which works by matching distributions between training and testing sets in feature space.
Cambridge University Press eBooksOn the method of bounded differences
1,428 Citations1989Colin McDiarmid
Proceedings of the National Academy of SciencesPredicting the clinical status of human breast cancer by using gene expression profiles
1,341 Citations2001Mike West, Carrie Blanchette +8 more
Bayesian regression models that provide predictive capability based on gene expression data derived from DNA microarray analysis of a series of primary breast cancer samples are developed and the utility and validity of such models in predicting the status of tumors in crossvalidation determinations are assessed.
Journal of the American Statistical AssociationProbability Inequalities for Sums of Bounded Random Variables
1,127 Citations1963Wassily Hoeffding
Learning and evaluating classifiers under sample selection bias
808 Citations2004Bianca Zadrozny
This paper formalizes the sample selection bias problem in machine learning terms and study analytically and experimentally how a number of well-known classifier learning methods are affected by it.
IEEE Transactions on Pattern Analysis and Machine IntelligenceBayesian classification with Gaussian processes
748 Citations1998Christopher K. I. Williams, David Barber
A Bayesian treatment is provided, integrating over uncertainty in y and in the parameters that control the Gaussian process prior the necessary integration over y is carried out using Laplace's approximation, and the method is generalized to multiclass problems (m>2) using the softmax function.
PubMedEstrogen receptor status in breast cancer is associated with remarkably distinct gene expression patterns.
682 Citations2001Sofia Gruvberger, Markus Ringnér +7 more
The results provide evidence that ER+ and ER- tumors display remarkably different gene-expression phenotypes not solely explained by differences in estrogen responsiveness, and could accurately predict ER status even when excluding top discriminator genes, including ER itself.
3 Statistical Inference
515 Citations2001Eric Y. Chai
Machine LearningSupport Vector Machines for Classification in Nonstandard Situations
348 Citations2002Yi Lin, Yoonkyung Lee +1 more
This paper explains why the standard support vectors machine is not suitable for the nonstandard situation, and introduces a simple procedure for adapting the support vector machine methodology to the non standard situation.
BMC BioinformaticsCross-platform analysis of cancer microarray data improves gene expression based classification of phenotypes
233 Citations2005Patrick Warnat, Roland Eils +1 more
Cross-platform classification of multiple cancer microarray data sets yields discriminative gene expression signatures that are found and validated on a large number of microarray samples, generated by different laboratories and microarray technologies.
Correcting sample selection bias in maximum entropy density estimation
205 Citations2005Miroslav Dudı́k, Steven J. Phillips +1 more
This work studies the problem of maximum entropy density estimation in the presence of known sample selection bias and proposes three bias correction approaches, which take advantage of unbiased sufficient statistics which can be obtained from biased samples.
Journal of ComplexitySupport Vector Machines are Universally Consistent
167 Citations2002Ingo Steinwart
It is shown that the 1-norm soft margin classifier with Gaussian RBF kernel on a compact subset X of Rd and regularization parameter cn = nβ-1 is universally consistent, if n is the training set size and 0 >β> 1/d.
Lecture notes in computer scienceA Few Notes on Statistical Learning Theory
163 Citations2003Shahar Mendelson
The focus in this article is on the theoretical side and not on the applicative one; hence, it shall not present examples which may be interesting from the practical point of view but have little theoretical significance.
Statistics & DecisionsInput-dependent estimation of generalization error under covariate shift
131 Citations2005Masashi Sugiyama, Klaus‐Robert Müller
This paper proposes an alternative estimator of the generalization error for the squared loss function when training and test distributions are different and is shown to be exactly unbiased for finite samples if the learning target function is realizable and asymptotically unbiased in general.
Speaker identification via support vector classifiers
107 Citations2002M. Schmidt, H. Gish
Though experimental results are preliminary, performance improvements over the BBN modified Gaussian Bayes decision system have been obtained on the Switchboard corpus.
A Method for Inferring Label Sampling Mechanisms in Semi-Supervised Learning
39 Citations2004Saharon Rosset, Ji Zhu +2 more
A method of moments for estimating this stochastic dependence using the unlabeled data in semi-supervised learning, where the "label sampling" mechanism stochastically depends on the true response (as well as potentially on the features).