visualize weights for colored images?

[charbelc]

Hello, Is there any other tutorial than MNIST? Is there a tutorial using colored images so that we can see how we can visualize the weights(and other functions) for colored images?

[charbelc]

In a general way, i would like to know if we can use the color information in these kinds of networks. I appreciate a lot your help

[BenjaminBossan]

I'm not too sure what you mean, do you want to use plot_conv_weight with a CNN that takes colored images? Because I think that should already work.

In general, if you want to use colored images, your data should have the shape samples x color channels x height x width. Color channels should be 3 for colored images. If you have that, you can use the same approach as in the tutorial.

[charbelc]

Thank you for your reply. plot_conv_weight generates 3 figures one for each channel, and when i try to combine them to see the RGB figures i get black images. Is there any tutorial using those functions(plot_conv_weight, plot_conv_activity...) ? for example on CIFAR dataset?

Thank you in advance

[BenjaminBossan]

I don't think plotting the weights in color will give you much additional insights. Why you get black images I can't tell but you should look at whether the values of the image data look reasonable.

Regarding tutorials for colored images, I don't know of any. But many Kaggler's used nolearn for computer vision competitions, maybe you can find something in the corresponding forums.

[charbelc]

The plot_conv_activity plots only one figure and in grayscale(even if the input is 3 channels), do you know why? Isn't it supposed to draw 3 figures, one for each channel?

[BenjaminBossan]

In general, plot_conv_activity plots exactly one image per channel/filter. The input layer has exactly 3 color channels, which you may plot as a colored image -- but this would just be the input image, so nothing is gained. The subsequent convolutional layers typically have at least 16 channels/filters, which makes plotting them in color useless. That is the reason why plot_conv_activity does not make special arrangements for colored images.

[dnouri]

FWIW, Krizhevsky 2012 has a plot with the filters learned in the first conv layer.

[DanChianucci]

I think this might be what you are looking for. It is similar to plot_conv_weights, except If the filters have 3 channels it will display them as RGB.

https://gist.github.com/DanChianucci/431b49de8f13789b039e239636cd12c5

conv_weights is expected to be in the shape (N , Ch, H, W)

• where N is the number of filters, Ch is the number of channels, HxW is the filter size.
• If there are more than 3 channels, the result is to sum them along the channel axis to create a greyscale representation

By default each filter is scaled between 0-1 by channel (global_scale==False).