Explain CNN+LSTM Classification

[gabarlacchi]

Hi! I am developing a clip classificator using features (extracted from a CNN) fed into an LSTM for classification. It's an end-to-end system. I would like to know if it's possible apply eli5 visualization tool to such a network in order to display, at each classification, the area into the images of the clip that has been "important" for classify if. Someone has already implemented such a system? Thanks in advance, Gabriele

[teabolt]

Hi @barloccia, ELI5 can highlight what's important in an image for Keras CNN's, https://eli5.readthedocs.io/en/latest/tutorials/keras-image-classifiers.html. Support for LSTM's and other libraries is still work in progress and not merged https://github.com/TeamHG-Memex/eli5/pull/325, https://github.com/TeamHG-Memex/eli5/pull/327. Regards, Tomas

[simplezhang57]

how to use "permutation_importance" on keras LSTM to find out feature importance?

[gabarlacchi]

@teabolt Thanks for the reply. I managed to make what desired with CONV-LSTM and visualize the activations through the time.