Hi, this tool is really cool! I am wondering why each filter in the second conv layer is connected to different activation maps in the previous layer?

[sudmat]

[aharley]

Great question! This is a re-implementation of a specific neural network from "Gradient-based learning applied to document recognition" by Lecun, Bottou, Bengio, and Haffner (1998), and the connection pattern was an implementation choice in that work. I've attached a screenshot from "Table 1" of that paper, describing the connection pattern, which I think should match what you see in the visualization.

The paper says this about it:

Why not connect every S2 feature map to every C3 feature map? The reason is twofold. First, a noncomplete connection scheme keeps the number of connections within reasonable bounds. More importantly, it forces a break of symmetry in the network. Different feature maps are forced to extract different (hopefully complementary) features because they get different sets of inputs