Question about dynamic routing with BatchSize > 1 for HAM10000 FixCaps model

[udiboy1209]

Hello! I am replicating your model for the HAM10000 for a course project. I had a question about the dynamic routing implementation inside your Digit Caps module.

In model.py line 120, you initialize the b_ij routing logits to have shape (1, features, num_units, 1). Here the routing logits have a batch dimension of 1, implying that one set of routing logits are shared across the entire batch. Similarly, on line 142 you have taken a mean of u_vj1 across dimension 0 (batch dimension).

It is my understanding that these routing logits should be different for each image in the batch. So the dimension of b_ij should actually be (batch, features, num_units, 1). Also you should not perform the mean(dim=0) on u_vj1. This averaging is a problem for me because, when I try to run the test dataset with batchsize 1 (T_size = 1), the accuracy drops to ~30%. This is generally not expected at testing time, because ideally each image's inference should be independent.

Question: I was wondering if you could tell me some details about why you have opted to average the dynamic routing logits across the batch?

Note that we did not face issue in recreating your accuracy with the correct batch size settings, based on the FixCaps_HAM-29.ipynb and your comments in issue #1 .

Thank you for the well commented code and detailed explanation of running your notebooks!