GradCam for unsupervised data

[michaelstuffer98]

I have a ViT-MAE that is pretrained in an unsupervised manner. I use a mask ratio of 80%, such that my latent (20% encoded patches) shape is [B, 14140.2+1, 768] . I use patches of size 16 with images of size 224 (224//16=14) and a class token. Since I try to use this auto encoder in a anomaly detection setting, is there a way to apply GradCam to this? The latent space is the output of the ViT encoder part and only consists of the 20% encoded patches which are decoded to the full image again.