Bayesian Networks and Decision Graphs
Information science and statisticsPublished 1 January 2007
Finn V. Jensen, Thomas D. Nielsen
Citations4,136
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
Probabilistic graphical models and decision graphs are powerful modeling tools for reasoning and decision making under uncertainty. As modeling languages they allow a natural specification of problem
Keywords
Computer Science
Journal of the Royal Statistical Society Series B (Statistical Methodology)Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
49,657 Citations1977A. P. Dempster, N. M. Laird +1 more
IEEE Transactions on Pattern Analysis and Machine IntelligenceStochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
17,980 Citations1984Stuart Geman, Donald Geman
The analogy between images and statistical mechanics systems is made and the analogous operation under the posterior distribution yields the maximum a posteriori (MAP) estimate of the image given the degraded observations, creating a highly parallel ``relaxation'' algorithm for MAP estimation.
American Mathematical Society eBooksTheory of games and economic behavior
16,943 Citations2019Stephan Ramon Garcia, Steven J. Miller
Machine LearningInduction of Decision Trees
14,815 Citations1986J. R. Quinlan
This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail, which is described in detail.
Pattern classification and scene analysis
12,643 Citations1973Richard O. Duda, Peter E. Hart
Journal of the American Statistical AssociationMarkov Decision Processes: Discrete Stochastic Dynamic Programming.
8,422 Citations1995Kasra Hazeghi, Martin L. Puterman
Markov Decision Processes covers recent research advances in such areas as countable state space models with average reward criterion, constrained models, and models with risk sensitive optimality criteria, and explores several topics that have received little or no attention in other books.
Lecture notes in statisticsCausation, Prediction, and Search
4,740 Citations1993Peter Spirtes, Clark Glymour +1 more
The authors axiomatize the connection between causal structure and probabilistic independence, explore several varieties of causal indistinguishability, formulate a theory of manipulation, and develop asymptotically reliable procedures for searching over equivalence classes of causal models.
Machine LearningBayesian Network Classifiers
4,738 Citations1997Nir Friedman, Dan Geiger +1 more
Tree Augmented Naive Bayes (TAN) is single out, which outperforms naive Bayes, yet at the same time maintains the computational simplicity and robustness that characterize naive Baye.
Journal of the Royal Statistical Society Series B (Statistical Methodology)Local Computations with Probabilities on Graphical Structures and Their Application to Expert Systems
3,970 Citations1988Steffen L. Lauritzen, David J. Spiegelhalter
This work exploits a range of local representations for the joint probability distribution, combined with topological changes to the original network termed 'marrying' and 'filling-in', which allows efficient algorithms for transfer between representations, providing rapid absorption and propagation of evidence.
Machine LearningLearning Bayesian Networks: The Combination of Knowledge and Statistical Data
3,242 Citations1995David Heckerman, Dan Geiger +1 more
A methodology for assessing informative priors needed for learning Bayesian networks from a combination of prior knowledge and statistical data is developed and how to compute the relative posterior probabilities of network structures given data is shown.
Machine LearningOn the Optimality of the Simple Bayesian Classifier under Zero-One Loss
3,072 Citations1997Pedro Domingos, Michael J. Pazzani
The Bayesian classifier is shown to be optimal for learning conjunctions and disjunctions, even though they violate the independence assumption, and will often outperform more powerful classifiers for common training set sizes and numbers of attributes, even if its bias is a priori much less appropriate to the domain.
IEEE Transactions on Information TheoryApproximating discrete probability distributions with dependence trees
2,640 Citations1968Chee Lap Chow, C. Liu
It is shown that the procedure derived in this paper yields an approximation of a minimum difference in information when applied to empirical observations from an unknown distribution of tree dependence, and the procedure is the maximum-likelihood estimate of the distribution.
Learning in Graphical Models
2,416 Citations1998Michael I. Jordan
Artificial IntelligenceFusion, propagation, and structuring in belief networks
2,152 Citations1986Judea Pearl
Artificial IntelligenceThe computational complexity of probabilistic inference using bayesian belief networks
1,935 Citations1990Gregory F. Cooper
It is shown that probabilistic inference using belief networks is NP-hard,Therefore, research should be directed away from the search for a general, efficient probabilism inference algorithm, and toward the design of efficient special-case, average- case, and approximation algorithms.
How to Solve It: Modern Heuristics
1,633 Citations2004Zbigniew Michalewicz, David B. Fogel
This book discusses computational experiments with heuristic methods for solving practical problems and basic concepts of probability and Statistics, as well as an Evolutionary approach to decision-making.
Operations ResearchEvaluating Influence Diagrams
1,276 Citations1986Ross D. Shachter
An algorithm is developed that can evaluate any well-formed influence diagram and determine the optimal policy for its decisions and can be performed using the decision maker's perspective on the problem.
ACM eBooksEquivalence and Synthesis of Causal Models
997 Citations2022TS Verma, Judea Pearl
The canonical representation presented here yields an efficient algorithm for determining when two embedded causal models reflect the same dependency information, which leads to a model theoretic definition of causation in terms of statistical dependencies.
IEEE Transactions on Systems Science and CyberneticsInformation Value Theory
973 Citations1966Ronald A. Howard
The theory of the value of information that arises from considering jointly the probabilistic and economic factors that affect decisions is discussed and illustrated and it is found that numerical values can be assigned to the elimination or reduction of any uncertainty.
Artificial IntelligenceLearning Bayesian networks from data: An information-theory based approach
803 Citations2002Jie Cheng, Russell Greiner +3 more
Algorithms that use an information-theoretic analysis to learn Bayesian network structures from data, requiring only polynomial numbers of conditional independence tests in typical cases are provided.
Journal of the ACMOn the Desirability of Acyclic Database Schemes
797 Citations1983Catriel Beeri, Ronald Fagin +2 more
It is shown that this class of database schemes, called acychc, has a number of desirable properties that have been studied by other researchers and are shown to be eqmvalent to acydicity.
Computational IntelligenceLEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE
773 Citations1994Wai Lam, Fahiem Bacchus
A new approach for learning Bayesian belief networks from raw data is presented, based on Rissanen's minimal description length (MDL) principle, which can learn unrestricted multiply‐connected belief networks and allows for trade off accuracy and complexity in the learned model.
Computational Statistics & Data AnalysisThe EM algorithm for graphical association models with missing data
763 Citations1995Steffen L. Lauritzen
It is shown how the computational scheme of Lauritzen and Spiegelhalter (1988) can be exploited to perform the E-step of the EM algorithm when applied to findingmaximum likelihood estimates or penalized maximum likelihood estimates in hierarchical log-linear models and recursive models for contingency tables with missing data.
ACM eBooksReverend Bayes on Inference Engines: A Distributed Hierarchical Approach
751 Citations2022Judea Pearl
Generalizations of Bayes likelihood-ratio updating rule are presented which facilitate an asynchronous propagation of the impacts of new beliefs and/or new evidence in hierarchically organized inference structures with multi-hypotheses variables.
BMJComputer-aided Diagnosis of Acute Abdominal Pain
674 Citations1972F T de Dombal, D J Leaper +3 more
It is suggested as a result of these studies that the provision of such a system to aid the clinician is both feasible in a real-time clinical setting, and likely to be of practical value, albeit in a small percentage of cases.
Machine intelligence and pattern recognitionPropagating Uncertainty in Bayesian Networks by Probabilistic Logic Sampling
552 Citations1988Max Henrion
Probabilistic logic sampling is a new scheme employing stochastic simulation which can make probabilistic inferences in large, multiply connected networks, with an arbitrary degree of precision controlled by the sample size.
NetworksSequential updating of conditional probabilities on directed graphical structures
533 Citations1990David J. Spiegelhalter, Steffen L. Lauritzen
It is shown how one can introduce imprecision into such probabilities as a data base of cases accumulates and how to take advantage of a range of well-established statistical techniques.
arXiv (Cornell University)Object-Oriented Bayesian Networks
497 Citations2013Daphne Koller, Avi Pfeffer
arXiv (Cornell University)Tractable Inference for Complex Stochastic Processes
488 Citations2013Xavier Boyen, Daphne Koller
IEEE Transactions on Knowledge and Data EngineeringA guide to the literature on learning probabilistic networks from data
477 Citations1996Wray Buntine
Journal of the Royal Statistical Society Series B (Statistical Methodology)On Use of the Em Algorithm for Penalized Likelihood Estimation
416 Citations1990Peter J. Green
Property of the EM algorithm in such contexts are discussed, concentrating on rates of conver- gence, and an alternative that is usually more practical and converges at least as quickly is presented.
Bucket Elimination: A Unifying Framework for Probabilistic Inference
384 Citations1998Rina Dechter
Probabilistic inference algorithms for finding the most probable explanation, the maximum aposteriori hypothesis, and the maximum expected utility and for updating belief are reformulated as an elimination-type algorithm called bucket elimination, emphasizing the principle common to many of the algorithms appearing in that literature.
Methods of Information in MedicineToward Normative Expert Systems: Part I The Pathfinder Project
371 Citations1992David Heckerman, Eric Horvitz +1 more
Machine intelligence and pattern recognitionCausal Networks: Semantics and Expressiveness
349 Citations1990Thomas Verma, Judea Pearl
This paper shows that the graphical criterion called d-separation is a sound rule for reading independencies from any DAG based on a causal input list drawn from a graphoid and may be extended to cover DAGs that represent functional dependencies as well as conditional dependencies.
Medical Decision MakingSensitivity Analysis and the Expected Value of Perfect Information
333 Citations1998James C. Felli, Gordon B. Hazen
The authors propose a fourth measure based upon the expected value of perfect information (EVPI), which they believe superior both methodologically and prag matically and provides a superior picture of problem sensitivity.
Defense Technical Information Center (DTIC)Bayesian Network Induction via Local Neighborhoods
316 Citations1999Dimitris Margaritis, Sebastian Thrun
This work presents an efficient algorithm for learning Bayes networks from data by first identifying each node's Markov blankets, then connecting nodes in a maximally consistent way, and proves that under mild assumptions, the approach requires time polynomial in the size of the data and the number of nodes.
Machine intelligence and pattern recognitionSimulation Approaches to General Probabilistic Inference on Belief Networks
311 Citations1990Ross D. Shachter, Mark A. Peot
This paper investigates a family of “forward” Monte Carlo sampling techniques similar to Logic Sampling which appear to perform well even in some multiplyconnected networks with extreme conditional probabilities, and thus would be generally applicable.
Machine intelligence and pattern recognitionWeighing and Integrating Evidence for Stochastic Simulation in Bayesian Networks
273 Citations1990Robert Fung, Kuo‐Chu Chang
The evidence weighting mechanism, for augmenting the logic sampling stochastic simulation algorithm, and an enhancement to the basic algorithm which uses the evidential integration technique [Chin and Cooper, 1987].
IEEE Transactions on Systems Man and CyberneticsDynamic programming and influence diagrams
268 Citations1990Joseph A Tatman, R.D. Shachter
By representing value function separability in the structure of the graph of the influence diagram, formulation is simplified and operations on the model can take advantage of the separability, this allows simple exploitation in the value function of a decision problem.
arXiv (Cornell University)Being Bayesian about Network Structure
267 Citations2013Nir Friedman, Daphne Koller
This paper shows how to efficiently compute a sum over the exponential number of networks that are consistent with a fixed ordering over network variables, and uses this result as the basis for an algorithm that approximates the Bayesian posterior of a feature.
IEEE Transactions on Systems Man and Cybernetics - Part A Systems and HumansSensitivity analysis in discrete Bayesian networks
264 Citations1997Enrique Castillo, José Manuel Gutiérrez +1 more
The proposed method can also be used to compute exact upper and lower bounds for the conditional probabilities, hence a sensitivity analysis can be easily performed.
Statistics and ComputingApplications of a general propagation algorithm for probabilistic expert systems
238 Citations1992A. P. Dawid
This paper analyses a ‘flow-propagation’ algorithm for calculating marginal and conditional distributions in a probabilistic expert system in detail, and shows how it can be modified to perform other tasks, including maximization of the joint density and simultaneous 'fast retraction' of evidence entered on several variables.
International Joint Conference on Artificial IntelligenceA computational model for causal and diagnostic reasoning in inference systems
236 Citations1983Jin Hee Kim, Judea Pearl
A representation of evidential relationships which permits updating of belief in two simultaneous modes: causal and diagnostic is introduced, which extends the hierarchical tree representation by allowing multiple causes to a given manifestation.
Operations ResearchValuation-Based Systems for Bayesian Decision Analysis
224 Citations1992Prakash P. Shenoy
A new method for representing and solving Bayesian decision problems is proposed, called a valuation-based system and has some similarities to influence diagrams, but unlike influence diagrams which emphasize conditional independence among random variables, valuation- based systems emphasize factorizations of joint probability distributions.
Elsevier eBooksA Bayesian Method for Constructing Bayesian Belief Networks from Databases
224 Citations1991Gregory F. Cooper, Edward H. Herskovits
A Bayesian method for constructing Bayesian belief networks from a database of cases, with potential applications include computer-assisted hypothesis testing, automated scientific discovery, and automated construction of probabilistic expert systems.
Communications of the ACMDecision-theoretic troubleshooting
216 Citations1995David Heckerman, John S. Breese +1 more
You have just finished typing that big report into your word processor, it is formatted correctly and looks beautiful on the screen, you hit print, go to the printer—and nothing is there.
Artificial IntelligenceLazy propagation: A junction tree inference algorithm based on lazy evaluation
216 Citations1999Anders L. Madsen, Finn V. Jensen
A junction tree based inference architecture exploiting the structure of the original Bayesian network and independence relations induced by evidence to improve the efficiency of inference is presented.
IEEE Transactions on Systems Man and CyberneticsSensitivity analysis for probability assessments in Bayesian networks
213 Citations1995Kathryn Blackmond Laskey
This paper presents a methodology for analytic computation of sensitivity values in Bayesian network models, which measure the impact of small changes in a network parameter on a target probability value or distribution.
Elsevier eBooksFrom Influence Diagrams to Junction Trees
212 Citations1994Frank Jensen, Finn V. Jensen +1 more
An approach to the solution of decision problems formulated as influence diagrams involves a special triangulation of the underlying graph, the construction of a junction tree with special properties, and a message passing algorithm operating on the junction tree for computation of expected utilities and optimal decision policies.
arXiv (Cornell University)Strong Completeness and Faithfulness in Bayesian Networks
199 Citations2013Christopher Meek
It is shown that in a strong measure-theoretic sense almost all discrete distributions for a given network structure are faithful; i.e. the independence facts true of the distribution are all and only those entailed by the network structure.
Uncertainty in Artificial IntelligenceA computational scheme for reasoning in dynamic probabilistic networks
179 Citations1992Uffe Kjærulff
The scheme is viewed as a generalization of the inference methods of classical time-series analysis in the sense that it allows description of non-linear, multivariate dynamic systems with complex conditional independence structures.
arXiv (Cornell University)Elicitation of Probabilities for Belief Networks: Combining Qualitative and Quantitative Information
177 Citations2013Marek J. Drużdżel, Linda C. van der Gaag
Computers and medicineExperience with a Model of Sequential Diagnosis
174 Citations1985G. Anthony Gorry, G. Octo Barnett
Journal of Philosophical LogicStochastic independence, causal independence, and shieldability
166 Citations1980Wolfgang Spohn
The aim of the paper is to explicate the concept of causal independence between sets of factors and Reichenbach's screening-off-relation in probabilistic terms along the lines of Suppes' probabilism theory of causality (1970).
arXiv (Cornell University)Making Sensitivity Analysis Computationally Efficient
163 Citations2013Uffe Kjærulff, Linda C. van der Gaag
This paper presents a method that requires just a single outward propagation in a junction tree for establishing the coefficients in the functions for all possible parameters; in addition, an inward propagation is required for processing evidence.
Local learning in probabilistic networks with hidden variables
146 Citations1995Stuart Russell, John Binder +2 more
It is shown that networks with fixed structure containing hidden variables can be learned automatically from data using a gradient-descent mechanism similar to that used in neural networks, which is extended to networks with intensionally represented distributions.
arXiv (Cornell University)Finding Optimal Bayesian Networks
112 Citations2012David Maxwell Chickering, Christopher Meek
This paper derives optimality results for greedy Bayesian-network search algorithms that perform single-edge modifications at each step and use asymptotically consistent scoring criteria and shows that the composition property is guaranteed to hold whenever the dependence relationships in the generative distribution can be characterized by paths between singleton elements in some generative graphical model.
Machine intelligence and pattern recognitionOn the Logic of Causal Models
110 Citations1990Dan Geiger, Judea Pearl
It is shown that DAGs offer polynomially sound and complete inference mechanisms for inferring conditional independence relationships from a given causal set of such relationships, and d-separation, a graphical criterion for identifying independencies in a DAG, is shown to uncover more valid independencies then any other criterion.
Explanation in Bayesian belief networks
86 Citations1992Henri J. Suermondt
This dissertation contributes a mathematical methodology that lets us determine the separate influences--on the belief-network inference result-- of individual findings, sets of findings, belief- network arcs, and chains of reasoning, which results in a set of functions that can be used to generate explanations.
Asymptotic Model Selection for Directed Networks with Hidden Variables
79 Citations1998Dan Geiger, David Heckerman +1 more
The Bayesian Information Criterion (BIC), an asymptotic approximation for tile marginal likelihood, is extended to Bayesian networks with hidden variables and it is argued that the dimension of a Bayesian uetwork withhidden variables is tile rank of the Jacobian matrix of the transformation between the parameters of the network and the parametersof the observable variables.
DSpace@MIT (Massachusetts Institute of Technology)Observation of a Markov process through a noisy channel
74 Citations1962Alvin W. Drake
Uncertainty in Artificial IntelligenceaHUGIN: A System Creating Adaptive Causal Probabilistic Networks
65 Citations1992Kristian G. Olesen, Steffen L. Lauritzen +1 more
A tool for creating adaptive systems, aHUGIN is an extension of the HUGIN shell, and is based on the methods reported by Spiegelhalter and Lauritzen (1990a), and the adaptive systems resulting from aHugIN are able to adjust the conditional probabilities in the model.
Management ScienceRepresentation and Solution of Decision Problems Using Sequential Decision Diagrams
59 Citations1995Zvi Covaliu, Robert M. Oliver
It is shown that a unified framework consisting of a sequential diagram, an influence diagram, and a common formulation table for the problem's data, suffices for compact and consistent representation, economical formulation, and efficient solution of (asymmetric) decision problems.
Journal of the Royal Statistical Society Series C (Applied Statistics)Updating a Diagnostic System Using Unconfirmed Cases
57 Citations1976D. M. Titterington
The general conclusion is that the discriminatory performance of the data bank can be usefully improved by making use of uncategorized observations.
Annals of Mathematics and Artificial IntelligenceLocal computation with valuations from a commutative semigroup
55 Citations1997Steffen L. Lauritzen, Finn V. Jensen
It is shown that propagation of belief functions can be performed in a HUGIN-like architecture or in the architecture of Lauritzen and Spiegelhalter.
arXiv (Cornell University)Lazy Propagation in Junction Trees
55 Citations2013Anders L. Madsen, Finn V. Jensen
Machine intelligence and pattern recognitionA Comparison of Decision Analysis and Expert Rules for Sequential Diagnosis
55 Citations1990Jayant Kalagnanam, Max Henrion
An experimental comparison of the performance of the two approaches to troubleshooting, specifically to test selection for fault diagnosis, using as experimental testbed the problem of diagnosing motorcycle engines suggests some interesting implications for knowledge acquisition.
International Journal of Approximate ReasoningPotential influence diagrams
50 Citations1994Pierre Ndilikilikesha
This study introduces potential influence diagrams, a generalization of standard influence diagrams in which each chance node is associated with an arbitrary nonnegative function (called a potential) instead of a conditional probability table, and develops a new reduction algorithm for computing optimal strategies.
arXiv (Cornell University)Myopic Value of Information in Influence Diagrams
46 Citations2013Sören Dittmer, Finn V. Jensen
Uncertainty in Artificial IntelligenceAnalysis in HUGIN of data conflict
46 Citations1990Finn V. Jensen, Bo Chamberlain +2 more
This paper presents a way of building a critical eye into a system with a CPN model and requires an easy way of calculating probabilities for specific configurations.
arXiv (Cornell University)Efficient Value of Information Computation
45 Citations2013Ross D. Shachter
arXiv (Cornell University)Lazy Evaluation of Symmetric Bayesian Decision Problems
43 Citations2013Anders L. Madsen, Finn V. Jensen
arXiv (Cornell University)Unconstrained Influence Diagrams
43 Citations2012Finn V. Jensen, Marta Vomlelová
The concept of GS-DAG is introduced: a DAG incurporating an optimal step-strategy for any instantiation, and a method for constructing GS- DAGs is given, and it is shown how to use a GS-dAG for determining an optimal strategy.
IEEE Transactions on ComputersMyopic Policies in Sequential Classification
42 Citations1978Ben‐Bassat
Several rules for feature selection in myopic policy are examined for solving the sequential finite classification problem with conditionally independent binary features, finding that no rule is consistently superior to the others.
arXiv (Cornell University)Conditions Under Which Conditional Independence and Scoring Methods Lead to Identical Selection of Bayesian Network Models
39 Citations2013Robert G. Cowell
It is argued that for complete data and a given node ordering this division is largely a myth, by showing that cross entropy methods for checking conditional independence are mathematically identical to methods based upon discriminating between models by their overall goodness-of-fit logarithmic scores.
Elsevier eBooksConflict and Surprise: Heuristics for Model Revision
32 Citations1991Kathryn Blackmond Laskey
This paper presents a set of decision theoretically motivated heuristics for diagnosing situations in which a model is likely to provide an inadequate representation of the process being modeled.
Lecture notes in computer scienceSensitivity analysis in Bayesian networks
31 Citations1995Finn V. Jensen, Søren H. Aldenryd +1 more
Sensitivity analysis is concerned with questions on how sensitive the conclusion is to the evidence provided, and the heart of sensitivity analysis is to compute probabilities for the hypotheses given various subsets of the evidence.
Electroencephalography and Clinical Neurophysiology/Evoked Potentials SectionDiagnostic function of the microhuman prototype of the expert system — MUNIN
30 Citations1992Steen Andreassen, Björn Falck +1 more
The prototype was restricted to a limited "Microhuman" anatomy with only 6 muscles and 8 nerves, and a corresponding limitation on the number of local nerve lesions, and it attempted to give a detailed description of the most important groups of generalized nerve and muscle disorders.
arXiv (Cornell University)Evaluating Influence Diagrams using LIMIDs
30 Citations2013Dennis K. Nilsson, Steffen L. Lauritzen
Lecture notes in computer scienceGradient Descent Training of Bayesian Networks
28 Citations1999Finn V. Jensen
This work introduces tools for resistance and damping to guide the direction of convergence, and uses them for a new adaptation method which can also handle situations where parameters in the network covary are concerned.
Data Archiving and Networked Services (DANS)Practicable sensitivity analysis of Bayesian belief networks
23 Citations1998Veerle M.H. Coupé, Linda C. van der Gaag
The graphical independence structure of a Bayesian belief network induces various properties that allow for reducing the computational burden of a sensitivity analysis, and it is shown that several analyses can be identified as being uninformative because the conditional probabilities under study cannot be placed in the network's output.
arXiv (Cornell University)Probabilistic Inference in Influence Diagrams
21 Citations2013Nevin L. Zhang
arXiv (Cornell University)On the Detection of Conflicts in Diagnostic Bayesian Networks Using Abstraction
11 Citations2013Young‐Gyun Kim, Marco Valtorta
An algorithm is implemented that takes as input an annotated diagnostic Bayesian network and constructs, without assistance, a bipartite network to be used as a straw model and it is shown that in some cases this straw model is better that the independent straw model of Jensen et al., the only other straw model for which a construction algorithm has been designed and implemented.
Lecture notes in statisticsRepresenting and Solving Asymmetric Decision Problems Using Valuation Networks
9 Citations1996Prakash P. Shenoy
A generalization of the valuation network representations and solution technique to enable efficient representation and solution of asymmetric decision problems is described.
…