How do we extend this to TimeDistributed Layers?

[sharathadavanne]

Hi, great package and examples. Thanks a lot for doing this. Currently, I am working with TimeDistributed multilabel output. How do we extend it to work on this?

[raghakot]

I am assuming the input is something like (batchsize, time_dims, input_dims, channels). The heatmap will simply be of the same shape which shows how the attention will vary over time_dims.

As for multi-label output. we have the ability to set multiple filter_indices. so filter_indices = [0, 5] would mean that outputs 0, 5 are on.

It would be really cool to have examples for these use-cases in examples/ folder. If your code is not confidential or proprietary, consider submitting an example PR. I am willing to work with you to get it working. Having a wide variety of examples/ is really the key rather than documentation to illustrate the various use-cases.

In either case, I can help you debug with any specific issues you might be having.

[sharathadavanne]

Hi @raghakot I have emailed you some test data and the model. If we can get it working, then we can post it as an example here.

[raghakot]

Cool. Looking at it.

[raghakot]

Minor update. With backprop_modifier='relu' in saliency, you do see something for weak_out. Does it make sense? This is using the deconv approach from matthew ziegler's paper.

I am still working on a solution for TimeDistributed nets.

[sivabala1994]

Hi @raghakot, Great work! it was really useful for getting insights about my project results. I am working on a time series based data. I am doing a RNN classification task and It will be great if you could share the technique on how to use Keras-vis for saliency analysis in RNN(LSTM) models.