Causal Feature Selection
Published 29 October 2007
Constantin Aliferis Isabelle Guyon
Citations82
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This chapter reviews techniques for learning causal relationships from data, in application to the problem of feature selection, and highlights the benefits that causal discovery may draw from recent developments in feature selection theory and algorithms.
Abstract
The present chapter makes an argument in favor of understanding and utilizing the notion of causality for feature selection: from an algorithm design perspective, to enhance interpretation, build robustness against violations of the i.i.d. assumption, and increase parsimony of selected feature sets; from the perspective of method characterization, to help uncover superfluous or artifactual selected features, missed features, and features not only predictive but also causally informative.
Keywords
Computer Science
Principles and practice of structural equation modeling
36,205 Citations2015Rex B. Kline
The book aims to provide the skills necessary to begin to use SEM in research and to interpret and critique the use of method by others.
EconometricaInvestigating Causal Relations by Econometric Models and Cross-spectral Methods
22,877 Citations1969Clive W. J. Granger
ScienceQuantitative Monitoring of Gene Expression Patterns with a Complementary DNA Microarray
9,597 Citations1995Mark Schena, Dari Shalon +2 more
A high-capacity system was developed to monitor the expression of many genes in parallel by means of simultaneous, two-color fluorescence hybridization, which enabled detection of rare transcripts in probe mixtures derived from 2 micrograms of total cellular messenger RNA.
Journal of Educational PsychologyEstimating causal effects of treatments in randomized and nonrandomized studies.
9,471 Citations1974Donald B. Rubin
Lecture notes in statisticsCausation, Prediction, and Search
4,740 Citations1993Peter Spirtes, Clark Glymour +1 more
The authors axiomatize the connection between causal structure and probabilistic independence, explore several varieties of causal indistinguishability, formulate a theory of manipulation, and develop asymptotically reliable procedures for searching over equivalence classes of causal models.
Machine LearningA Bayesian Method for the Induction of Probabilistic Networks from Data
3,525 Citations1992Gregory F. Cooper, Edward H. Herskovits
This paper presents a Bayesian method for constructing probabilistic networks from databases, focusing on constructing Bayesian belief networks, and extends the basic method to handle missing data and hidden variables.
Journal of Computational BiologyUsing Bayesian Networks to Analyze Expression Data
3,320 Citations2000Nir Friedman, Michal Linial +2 more
Artificial IntelligenceSelection of relevant features and examples in machine learning
3,295 Citations1997Avrim Blum, Pat Langley
This survey reviews work in machine learning on methods for handling data sets containing large amounts of irrelevant information and describes the advances that have been made in both empirical and theoretical work in this area.
University of Maribor digital library (University of Maribor)Structural Equation Modeling: Foundations and Extensions
1,835 Citations2000David E. Kaplan
Studies in computational intelligenceA Tutorial on Learning with Bayesian Networks
1,699 Citations2008David Heckerman
Methods for constructing Bayesian networks from prior knowledge are discussed and methods for using data to improve these models are summarized, including techniques for learning with incomplete data.
Toward optimal feature selection
1,444 Citations1996Daphne Koller, Mehran Sahami
An efficient algorithm for feature selection which computes an approximation to the optimal feature selection criterion is given, showing that the algorithm effectively handles datasets with a very large number of features.
Journal of the American Statistical AssociationFeature Extraction Construction and Selection: A Data Mining Perspective
722 Citations1999MTW, Huan Liu +1 more
This book can be used by researchers and graduate students in machine learning, data mining, and knowledge discovery, who wish to understand techniques of feature extraction, construction and selection for data pre-processing and to solve large size, real-world problems.
Feature selection for high-dimensional genomic microarray data
629 Citations2001Eric P. Xing, Michael I. Jordan +1 more
This paper reports on the successful application of feature selection methods to a classification problem in molecular biology involving only 72 data points in a 7130 dimensional space and investigates regularization methods as an alternative to feature selection.
TechnometricsProbabilistic Reasoning in Expert Systems, Theory and Algorithms
616 Citations1992Peter M. Jones, Richard E. Neapolitan
This text is a reprint of the seminal 1989 book Probabilistic Reasoning in Expert systems: Theory and Algorithms, which helped serve to create the field the authors now call Bayesian networks and provides an insightful comparison of the two most prominent approaches to probability.
ePrints Soton (University of Southampton)Result Analysis of the NIPS 2003 Feature Selection Challenge
546 Citations2004Isabelle Guyon, Steve Gunn +2 more
The NIPS 2003 workshops included a feature selection competition organized by the authors, which took place over a period of 13 weeks and attracted 78 research groups and used a variety of methods for feature selection.
arXiv (Cornell University)Learning Bayesian Network Structure from Massive Datasets: The "Sparse Candidate" Algorithm
522 Citations1999Nir Friedman, Iftach Nachman +1 more
An algorithm that achieves faster learning by restricting the search space, which restricts the parents of each variable to belong to a small subset of candidates and is evaluated both on synthetic and real-life data.
Algorithms for Large Scale Markov Blanket Discovery
442 Citations2003Ioannis Tsamardinos, Constantin Aliferis +1 more
A low-order polynomial algorithm and several variants that soundly induce the Markov Blanket under certain broad conditions in datasets with thousands of variables are introduced and compared to other state-of-the-art local and global methods with excellent results.
Defense Technical Information Center (DTIC)Bayesian Network Induction via Local Neighborhoods
316 Citations1999Dimitris Margaritis, Sebastian Thrun
This work presents an efficient algorithm for learning Bayes networks from data by first identifying each node's Markov blankets, then connecting nodes in a maximally consistent way, and proves that under mild assumptions, the approach requires time polynomial in the size of the data and the number of nodes.
PubMedHITON: a novel Markov Blanket algorithm for optimal variable selection.
296 Citations2003Constantin Aliferis, Ioannis Tsamardinos +1 more
A novel, sound, sample-efficient, and highly-scalable algorithm for variable selection for classification, regression and prediction called HITON, which reduces the number of variables in the prediction models by three orders of magnitude relative to the original variable set while improving or maintaining accuracy.
Uncertainty in Artificial IntelligenceParameter adjustment in Bayes networks. The generalized noisy OR–gate
229 Citations1993Francisco Javier Díez
This work generalizes the noisy OR-gate for multivalued variables, develops the algorithm to compute probability in time proportional to the number of parents (even in networks with loops) and applies the learning model to this gate.
Causal Explorer: A Causal Probabilistic Network Learning Toolkit for Biomedical Discovery.
99 Citations2003Constantin Aliferis, Ioannis Tsamardinos +2 more
A software library that implements a suit of global, local and partial CPN induction algorithms, and emphasizes causal discovery algorithms that scale to thousands of variables and thus are particularly suitable for modeling in massive datasets.
GeNIeRate: An Interactive Generator of Diagnostic Bayesian Network Models
48 Citations2005Pieter Kraaijeveld
A methodology to simplify and speed up the design of very large Bayesian network models consisting of hundreds or even thousands of variables is proposed, and preliminary qualitative evaluation of GeNIeRate shows great promise.
Causal Inference by Choosing Graphs with Most Plausible Markov Kernels
46 Citations2006Xiaohai Sun, Dominik Janzing +1 more
This paper discusses the inference rule for causal relationships between two variables in detail, applies it to a real-world temperature data set with known causality and shows that the method provides a correct result for the example.
PubMedChallenges in the analysis of mass-throughput data: a technical commentary from the statistical machine learning perspective.
29 Citations2007Constantin Aliferis, Alexander Statnikov +1 more