Is mutual information transitive? Suppose A, B and C are random variables. Given that the mutual information between A and B is very large and also the mutual information between B and C is very large, could we conclude that the mutual information between A and C is also large? In other words, if we know the mutual information between A and B and the mutual information between B and C, what could we say about the mutual information between A and C? Thanks,

The relation between $I(A;C)$ and $I(A;B)$,$I(B;C)$ depends on the joint probability distribution of $A,B,C$. For instance if you have the Markov relation that $P(A|BC)=P(A|B)$, then this will imply that: $$ I(A;B)=I(A;BC)\geq I(A;C). $$ Similarly $$ I(B;C)=I(BA;C)\geq I(A;C). $$ Therefore $$ \min\left(I(A;B),I(B;C)\right)\geq I(A;C). $$ On the other hand if you assume different Markov relation as $P(A|BC)=P(A|C)$, you get: $$ I(A;C)=I(A;BC)\geq I(A;B). $$ So there is no specific relation between mutual informations if you do not know their joint distribution.