Gradcam for semantic/ instance segmentation model

[abhigoku10]

@yizt Thanks for sharing your work . How can we use the current code to for segmentation models like deeplab . Can you share / suggest the modifications required

Thanks in advance

[yizt]

@abhigoku10 您好，语义分割已经是像素级别结果了，不需要生成CAM图像

[abhigoku10]

@abhigoku10 wokay for architectures like retina net how to obtain the CAM output or mask rcnn based architecture

[yizt]

@abhigoku10 您好，retina net完成了，通知您哈

[abhigoku10]

@yizt please do let me knw once done it

[wangzyon]

@yizt 在线等前辈的retinanet CAM，因为retinanet score和box是分开的，自己一直没有搞出来

[yizt]

您好，我尽快，最迟这周末！

| | 易作天 | | 邮箱：[email protected] |

签名由 网易邮箱大师 定制

On 05/12/2020 11:03, wangzyon wrote:

@abhigoku10 您好，retina net完成了，通知您哈

@abhigoku10 您好，语义分割已经是像素级别结果了，不需要生成CAM图像

@yizt 在线等前辈的retinanet CAM，因为retinanet score和box是分开的，自己一直没有搞出来

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[yizt]

@abhigoku10 @wangzyon 已增加retinanet的Grad-CAM实现

[abhigoku10]

@yizt thanks for sharing the source code , in your codes both faster rcnn and retina net you have a variable called indices can you let me knw how to do it for custom architecture or for yolact

[yizt]

@abhigoku10 你好，a)个人认为跟indices没有关系，假如retinanet没有使用fpn，也就不需要indices了。对于目标检测网络由于是多输出，且网络结构都不同，没有固定的生成CAM的方法；b)另外yolact是实例分割网络，不需要生成CAM图了，分割结果就是最好的CAM图。c) 如果实在要生成yolact的CAM图，与retinanet应该是一样的，yolact是基于retinanet的

[abhigoku10]

@yizt hwo to get output like this from the above source code

[yizt]

@abhigoku10 您好，这就是原始图像与热图加权求和结果；如main.py中的

cam = heatmap + np.float32(image)
return norm_image(cam)

[abhigoku10]

@yizt i have following few queries

1. how to obtain the heatmap for all the classes in a single image / all the objects of the same class in a single image
2. what is the feature_level variable in the retinanet.py in singleinference() function

[yizt]

@abhigoku10 您好

1. 对于retinanet，没有办方获取同一类别所有对象；不同的对象(object)可能来自不同的feature level
2. retinanet使用了feature pyramid；feature level指的是对象对应的anchor来自feature pyramid中哪一层feature map;或者说预测该对象使用的是哪一层feature map

[abhigoku10]

@yizt i am trying to modify the code to run grad cam on the yolact++/yolact, i am stuck at the feature level function where i am not able to get generate the feature level values for this architecture, How to integrate with other architecture