Integrated gradients for nlp models. (Explainable AI)

[innat]

Describe feature The Integrated Gradients (IG) can be a great tool to understand the neural network.

How API will change? It would enhance as follows:

from keras_nlp.utils.visualization import IntegratedGradients as IG 

ig  = IG (model, layer=layer, ...)

Candidate Solution

• ALIBI-Explain: It's a toolbox that provides many solutions, including IG in TensorFlow 2 (Keras).

Demo o/p:

ig.explain(sample)

[mattdangerw]

Definitely, it would be great to start adding explainability tools here! I was actually just wishing we had precisely this while working on a guide for keras-nlp.

Would you be interested in contributing?

Also, trying to understand the design proposed more...

• IG is it's own non-layer class?
• What assumptions are being made about the model parameter passed in? If any.

[abheesht17]

Hey, @innat! Are you still working on this? If not, I'd like to work on this. Thanks! :)

[innat]

@abheesht17 I'm not working on it. Please, take action if you want.

[abheesht17]

Thanks, @innat! @mattdangerw, I can take this up (if we want this layer) :)