Should model output be class probabilities (should model end with softmax)?

[vedal]

❓ Questions and Help

In most docstrings of captum attribution methods, there is an example where the model is said to return class probabilities:

>>> # ImageClassifier takes a single input tensor of images Nx3x32x32,
>>> # and returns an Nx10 tensor of class probabilities.

However, the example net in captum tutorials such as this don't seem to have an nn.Softmax at the end. I also noted this issue, where both with and without softmax is suggested.

Finally, I noted that only captum.AttributionVisualizer mentions of a score function like softmax.

My question is: what is common practice: with or without softmax at the end of the network when computing attributions?

Edit: A summary paper section VI.C discussing this issue.