R Devon Hjelm
R Devon Hjelm
So a student of mine did this computing the mutual information between the global of one image with the locals of another with additional visualization. They should be making the...
So I didn't add it to this repo, but I can this week. It doesn't perform as well as dot product.
Yeh, it's just binary classification between pairs (global, local that comes from same image that produced global) and (global, local that comes from probably different image that that that produced...
So one quick question: the results that you're focusing on are training DIM and the classifier together, correct?
So the `encoder` is where everything is happening, from encoding the MI to scoring pairs from the joint / product of marginals. Unfortunately it's not separable in a simple way...
Did you do `return_rkhs=True` and `return_all_activations=True`? https://github.com/rdevon/DIM/blob/master/cortex_DIM/models/controller.py#L77
So when you pass data through the `Controller.encoder` network with those arguments, you should get a tuple of dictionaries. Is this the case?
ok, so the list should be all activations, the dict should have values that correspond to the local / global vectors from DIM.
Yes, and thanks for asking! I'll try to get this out soon, hopefully in 2 weeks time.
I'm going to leave this open until it's pushed :)