gen2-maskrcnn-resnet50 OUT_OF_MEMORY

[pinolo7]

Hi @Erol444,

thanks for all the different examples you've shared. It was incredible to see mask R-CNN run on this device, as I didn't think that was possible at all.

It runs everything fine when I use the pretrained openvino zoo model NN_PATH = str(blobconverter.from_zoo("mask_rcnn_resnet50_coco_300x300", shaves=6, zoo_type="depthai")) as showed in your main.py example, while it gives always out of memory when I try to convert it myself following the instructions

I don't understand how this happens, as the size of my model is around 92mb, like the one downloaded from your example, but your runs smootly and my gives error.

I've converted as showed in the instructions, the pytorch model maskrcnn resnet50 using Openvino 2021.4 and then to blob onhttps://blobconverter.luxonis.com/

 `import` torchvision

# Create model
model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True)

# Convert to OpenVINO IR
import mo_pytorch
img_size = 300
mo_pytorch.convert(model,
                   input_shape=[1, 3,img_size, img_size],
                   model_name="maskrcnn",
                   scale=255,
                   reverse_input_channels=True)`

Is probably there a parameter that I'm missing when converting it into blob, to make it lighter, or something else?

thank you

[tersekmatija]

Hi @pinolo7 ,

could you try adding box_detections_per_img=25 and rpn_post_nms_top_n_test=100 arguments when calling maskrcnn_resnet50_fpn. This reduces default number of detections per image from 100 to 25, as well as reduces number of proposals for NMS, which should reduce the memory consumption and not cause the error. I think I used those parameters during the conversion. If that works, I'll update the README.

[pinolo7]

Hi @tersekmatija ,

thank you for the suggestions. I've tried it and unfortunately the result doesn't change, I keep getting the same error.

This is the code I used.

model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True, box_detections_per_img=25, rpn_post_nms_top_n_test=100)

and it goes out of memory in the exact same way as before. Error: [18443010E119B60F00] [1116.973] [NeuralNetwork(3)] [warning] Network compiled for 4 shaves, maximum available 13, compiling for 6 shaves likely will yield in better performance [18443010E119B60F00] [1117.048] [NeuralNetwork(3)] [error] Neural network executor '1' out of '2' error: OUT_OF_MEMORY [18443010E119B60F00] [1117.051] [NeuralNetwork(3)] [error] Neural network executor '0' out of '2' error: OUT_OF_MEMORY [18443010E119B60F00] [1117.051] [NeuralNetwork(3)] [error] Not enough memory to start a given neural network, needed at least: 259709824B

I wonder if there is any parameter that I can change on the Myriad blob converter here https://blobconverter.luxonis.com/

I'm using the default settings there: -ip U8

If anyone has some idea of parameters that I can tweak, I'll be happy to try them. It would be ideal to know how the model in this demo https://github.com/luxonis/depthai-experiments/tree/master/gen2-maskrcnn-resnet50 was generated, because that one works very well in terms of accuracy and speed.

[tersekmatija]

Thanks, I am pretty sure those were the settings I used for export, but I'll check whether I still have the export script that I used here and get back to you.

Are you running anything else in your pipeline or are you using the script from the experiment?

[tersekmatija]

I am looking at the XML from the experiment, I think the correct settings should be box_detections_per_img=20, rpn_post_nms_top_n_test=50. Could you try this and share the XML of the model if it doesn't work? @pinolo7

[pinolo7]

I'm using the exact same script from the experiment, just changing the line where I load the model to load my converted model instead the one from zoo.

I tried also box_detections_per_img=20, rpn_post_nms_top_n_test=50 but it seems to have no effect at all, as the error displayed is exactly the same and also the memory requirement is the same, no matter how I change the values.

Not enough memory to start a given neural network, needed at least: 259709824B

I'm attaching here the xml file maskrcnn_r50_300.zip

thank you

[tersekmatija]

Hey, I noticed some discrepancies in the Detect layer when comparing yours and the one from our ZOO. Will have to investigate.

Best, Matija

[tersekmatija]

Hey, I re-checked this and I can confirm this was my export script:

import torchvision.models as models
import torch

# Create model
model = models.detection.mask_rcnn.maskrcnn_resnet50_fpn(pretrained=True, box_detections_per_img=20, rpn_post_nms_top_n_test=50)
model.eval()
print(model(torch.rand([1, 3, 300, 300])))

# Convert to OpenVINO IR
import mo_pytorch
mo_pytorch.convert(model, input_shape=[1, 3, 300, 300], model_name='maskrcnn_resnet50_fpn', scale = 255)