Extract features from various layers of MaskRCNN, such as ROI align features, pyramid features [P1,P2,P3,P4,P5]

[peymanrah]

I need to receive the feature maps of every layer of mask RCNN to be used as a deep feature for other classifier. How do I go about it? Any help or code is much appreciated!

[simonyang0608]

you mean the output featuremap in every layer of mrcnn architecture?

[peymanrah]

Yes, Think of it like VGG16/19 where you can extract the deep features from each layer and feed them to another ML model to be trained on. Here, we need to be able to extract Mask RCNN deep features at any layer of interest (RPN, Pyramid features, ROI align features, and etc.) How do we do this for Mask RCNN?

Any help on a block of code is much appreciated,

Thanks,

[9thDimension]

I'd like to be able to get the ROI-pooled features for each final detection and do extra analysis with them. I managed to get the net to return 100 ROI-pooled features, but I don't know which ones correspond to the final detection boxes.

I think you can modify the method fpn_classifier_graph() to return the result x from PyramidROIAlign. https://github.com/matterport/Mask_RCNN/blob/a8e674cc08fdab24290e39b331fad53a7c1e8600/mrcnn/model.py#L925

This x is the many (thousands?) ROI pooled features having all been resized to a fixed spatial size = 7x7x256.

However the subsequent method refine_detections_graph does some stuff I don't quite understand to reduce the thousand or so ROIs to just config.MAX_DETECTION_INSTANCS and possibly re-orders them. So I don't know how to choose the correct 7x7x256 tensor from the list.

[VellalaVineethKumar]

@peymanrah Hey, were you able to do this? if so please help me understand how you extracted features from the maskrcnn ?

[SwetaKaman]

@VellalaVineethKumar Hey, are you able to extract features of the detected bboxes??

[VellalaVineethKumar]

@SwetaKaman No I'm still searching for it. please let me know if you get is somewhere

[SwetaKaman]

Okay I understood how to modify the fpn_classifier_graph() method. But not getting the how to modify the refine_detections_graph() method.

On Wed, 10 Apr 2019, 8:40 pm V Vineeth kumar <[email protected] wrote:

@SwetaKaman https://github.com/SwetaKaman No I'm still searching for it. please let me know if you get is somewhere

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/matterport/Mask_RCNN/issues/1153#issuecomment-481731461, or mute the thread https://github.com/notifications/unsubscribe-auth/Ad0I8lo2lwtWoqBI7e6aGEXKd-Le6_qaks5vff7MgaJpZM4Y8apP .

[SwetaKaman]

@VellalaVineethKumar Send me your email address I'll send you my modified model.py just check the output

[ApoorvaSuresh]

Hi, @SwetaKaman @VellalaVineethKumar were you successful in doing this? please let me know

[TrungThanhTran]

Do you have any solutions for this issues, even in Faster R-CNN? We want to get ROI features but RPN PyramidROIAlign returns all ROI features, right. So we should choose the one has the biggest probability.

[tgupt]

Hi, @SwetaKaman can you pls let me know how did you go about it?

[Monteleone]

@VellalaVineethKumar Send me your email address I'll send you my modified model.py just check the output

@SwetaKaman Can i give you my email? Can you send me your code? I need the features map too

[rupa1118]

@VellalaVineethKumar Send me your email address I'll send you my modified model.py just check the output

Hey @SwetaKaman
Can you please send me it too ???? I too need to visualize the feature maps.

[ariefbros]

@SwetaKaman can you send me too?? Please.

[Unmicool]

@SwetaKaman could you please share the feature map visualisation process in mrcnn? Thank you in advance!