Pairwise Mutual Information

[N-McA]

Hey,

Are you sure the computation of pairwise mutual information in the hashing example is correct? I've been working through the maths (and reverse-engineering what you compute) and I'm not entirely convinced that the value you compute is correct.

You seem to be evaluating p(Y_a, Y_b | X) * lg (p(Y_a, Y_b | X) / p(Y_a, Y_b))

Whereas I was expecting the correct term to be I(Y_a ; Y_b) = H(Y_a) - H(Y_a | Y_b) =p(Y_a) * lg(p(Y_a)) - p(Y_a, Y_b) * lg (p(Y_a, Y_b) / p(Y_b))

I'm not certain yet though. Do you have any notes on how you derived the current form?