How to visualize a binary classification model using GradCam

[Minxiangliu]

trafficstars

Hi @FrancescoSaverioZuppichini, I have used the EfficientNet_B2 model from torchvision and modified the input and output layers for binary classification of grayscale images. Now, I want to visualize the areas of interest in the model using GradCam. However, it seems that your function does not provide implementation for binary classification (or I might be using it incorrectly). I encountered the following error. Can you please help me? Thanks in advance!

Package Version: torch 2.1.0 torchvision 0.16.0

Modify Source Code:

• GradCam.py

# predictions = self.module(input_var)
predictions = torch.nn.Sigmoid()(self.module(input_var))

• utils.py

def tensor2cam(image, cam):
#  image_with_heatmap = image2cam(image.squeeze().permute(1,2,0).cpu().numpy(), cam.detach().cpu().numpy())
    image_with_heatmap = image2cam(image.unsqueeze(-1).cpu().numpy(), cam.detach().cpu().numpy())

Modify Model Layer:

from torchvision.models import efficientnet_b2
from torchvision.models import EfficientNet_B2_Weights
from torchvision.models._api import WeightsEnum
from torch.hub import load_state_dict_from_url
def get_state_dict(self, *args, **kwargs):
    kwargs.pop("check_hash")
    return load_state_dict_from_url(self.url, *args, **kwargs)
WeightsEnum.get_state_dict = get_state_dict

model = efficientnet_b2(weights=EfficientNet_B2_Weights.IMAGENET1K_V1)
model.features[0][0] = torch.nn.Conv2d(1, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
model.classifier[1] = torch.nn.Linear(in_features=1408, out_features=1, bias=True)

Implementation:

import torch
from visualisation.core import GradCam
from visualisation.core.utils import device

input_batch = torch.rand((1,1,256,256))
vis = GradCam(model, device)

outs = vis(
    input_image=input_batch.cuda(), 
    layer=None, 
    guide=False,
    target_class=None,
    regression=False)

I'm getting cam that are all zeros.

[FrancescoSaverioZuppichini]

I mate, I am sorry but it has been a long time since I develop this package. From the top of my mind I have no idea why