Asymptotic Model Selection for Directed Networks with Hidden Variables
Published 1 January 1998
Dan Geiger, David Heckerman, Christopher Meek
Citations79
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The Bayesian Information Criterion (BIC), an asymptotic approximation for tile marginal likelihood, is extended to Bayesian networks with hidden variables and it is argued that the dimension of a Bayesian uetwork withhidden variables is tile rank of the Jacobian matrix of the transformation between the parameters of the network and the parametersof the observable variables.
Abstract
We extend the Bayesian Information Criterion (BIC), an asymptotic approximation for the marginal likelihood, to Bayesian networks with hidden variables. This approximation can be used to select models given large samples of data. The standard BIC as well as our extension punishes the complexity of a model according to the dimension of its parameters. We argue that the dimension of a Bayesian network with hidden variables is the rank of the Jacobian matrix of the transformation between the parameters of the network and the parameters of the observable variables. We compute the dimensions of several networks including the naive Bayes model with a hidden root node.This manuscript was previously published in The Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence, 1996, Morgan Kaufmann.
Keywords
Computer ScienceMathematics
Machine LearningA Bayesian Method for the Induction of Probabilistic Networks from Data
3,525 Citations1992Gregory F. Cooper, Edward H. Herskovits
This paper presents a Bayesian method for constructing probabilistic networks from databases, focusing on constructing Bayesian belief networks, and extends the basic method to handle missing data and hidden variables.
Machine LearningLearning Bayesian Networks: The Combination of Knowledge and Statistical Data
3,242 Citations1995David Heckerman, Dan Geiger +1 more
A methodology for assessing informative priors needed for learning Bayesian networks from a combination of prior knowledge and statistical data is developed and how to compute the relative posterior probabilities of network structures given data is shown.
Machine LearningA Bayesian method for the induction of probabilistic networks from data
2,344 Citations1992Gregory F. Cooper, Edward H. Herskovits
This paper presents a Bayesian method for constructing probabilistic networks from databases, focusing on constructing Bayesian belief networks, and extends the basic method to handle missing data and hidden variables.
Machine LearningLearning Bayesian networks: The combination of knowledge and statistical data
2,043 Citations1995David Heckerman, Dan Geiger +1 more
It is shown that likelihood equivalence when combined with previously made assumptions implies that the user's priors for network parameters can be encoded in a single Bayesian network for the next case to be seen—aprior network—and a single measure of confidence for that network.
Bayesian classification (AutoClass): theory and results
972 Citations1996Peter Cheeseman, John Stutz
It is emphasized that no current unsupervised classi(cid:12)cation system can produce maximally useful results when operated alone, and that it is the interaction between domain experts and the machine searching over the model space, that generates new knowledge.
Statistical ScienceBayesian Analysis in Expert Systems
687 Citations1993David Spiegelhalter, A. P. Dawid +2 more
Using a real, moderately complex, medical example, it is illustrated how qualitative and quantitative knowledge can be represented within a directed graphical model, generally known as a belief network in this context.
arXiv (Cornell University)Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence (1995)
554 Citations2013Philippe Besnard, Steve Hanks
Journal of Artificial Intelligence ResearchOperations for Learning with Graphical Models
538 Citations1994Wray Buntine
The main original contributions here are the decompositiontechniques and the demonstration that graphical models provide a framework for understanding and developing complex learning algorithms.
NetworksSequential updating of conditional probabilities on directed graphical structures
533 Citations1990David J. Spiegelhalter, Steffen L. Lauritzen
It is shown how one can introduce imprecision into such probabilities as a data base of cases accumulates and how to take advantage of a range of well-established statistical techniques.
Artificial IntelligenceConnectionist learning of belief networks
532 Citations1992Radford M. Neal
The “Gibbs sampling” simulation procedure for “sigmoid” and “noisy-OR” varieties of probabilistic belief networks can support maximum-likelihood learning from empirical data through local gradient ascent.
IEEE Transactions on Knowledge and Data EngineeringA guide to the literature on learning probabilistic networks from data
477 Citations1996Wray Buntine
Journal of the Royal Statistical Society Series B (Statistical Methodology)Stochastic Complexity
343 Citations1987J. Rissanen
Journal of Artificial Intelligence ResearchMean Field Theory for Sigmoid Belief Networks
339 Citations1996L.K. Saul, Tommi Jaakkola +1 more
The utility of a mean field theory for sigmoid belief networks based on ideas from statistical mechanics is demonstrated on a benchmark problem in statistical pattern recognition-the classification of handwritten digits.
The Annals of StatisticsOn the Choice of a Model to Fit Data from an Exponential Family
274 Citations1988Dominique Haughton
Utrecht University Repository (Utrecht University)Bayesian belief networks : from construction to inference
217 Citations1995Remco Bouckaert
Uncertainty in Artificial IntelligenceA Construction of Bayesian Networks from Databases Based on an MDL Principle
149 Citations1993Joe Suzuki
The extended version of the algorithm of Chow and Liu in that the learning algorithm selects the model in the range where the dependencies among the attributes are represented by some general plural number of trees.
Cambridge University Press eBooksA Bayesian Approach to Learning Causal Networks
142 Citations2007David Heckerman
This paper introduces two sufficient assumptions, called mechanism independence and component independence, and shows that these new assumptions, when combined with parameter independence, parameter modularity, and likelihood equivalence, allow methods for learning acausal networks to learn causal networks.
Computer-based probabilistic-network construction
117 Citations1992Edward H. Herskovits
This dissertation demonstrates that nonparametric, efficient, computer-based algorithms for determining the important associations among variables in a domain are conceptually feasible, robust to noise, computationally efficient, theoretically sound, and that they generate models that can classify new cases accurately.
Uncertainty in Artificial IntelligenceUsing Causal Information and Local Measures to Learn Bayesian Networks
65 Citations1993Wai Lam, Fahiem Bacchus
A new local way of computing the description length is presented, which allows for significant improvements in the search algorithm and opens the door for local refinement of an existent network.
arXiv (Cornell University)Efficient Approximations for the Marginal Likelihood of Incomplete Data Given a Bayesian Network
56 Citations2013David Maxwell Chickering, David Heckerman
A Characterization of the Dirichlet Distribution with Application to Learning Bayesian Networks
54 Citations1996Dan Geiger, David Heckerman
This work provides a new characterization of the Dirichlet distribution that implies that under assumptions made by several previous authors for learning belief networks, aDirichlet prior on the parameters is inevitable.
arXiv (Cornell University)Proceedings of the Twelfth Conference on Uncertainty in Artificial\n Intelligence (1996)
49 Citations2013Eric Horvitz, Finn V. Jensen
NASA Technical Reports Server (NASA)A Guide to the Literature on Learning Graphical Models
28 Citations1994Wray Buntine, Peter Friedland
This literature review discusses different methods under the general rubric of learning Bayesian networks from data, and more generally, learning probabilistic graphical models.