Information-geometric approach to inferring causal directions

This work defines independence via orthogonality in information space so that it can explicitly describe the kind of dependence that occurs between P"Y and P"X"|"Y making the causal hypothesis ''Y causes X'' implausible.

Abstract

While conventional approaches to causal inference are mainly based on conditional (in)dependences, recent methods also account for the shape of (conditional) distributions. The idea is that the causal hypothesis “X causes Y ” imposes that the marginal distribution P X and the conditional distribution PY |X represent independent mechanisms of nature. Recently it has been postulated that the shortest description of the joint distribution P X,Y should therefore be given by separate descriptions of P X and PY |X . Since description length in the sense of Kolmogorov complexity is uncomputable, practical implementations rely on other notions of independence. Here we define independence via orthogonality in information space. This way, we can explicitly describe the kind of dependence that occurs between PY and P X|Y making the causal hypothesis “Y causes X” implausible. Remarkably, this asymmetry between cause and effect becomes particularly simple if X and Y are deterministically related. We present an inference method that works in this case. We also discuss some theoretical results for the non-deterministic case although it is not clear how to employ them for a more general inference method.