Inference with CPU only

[DangChuong-DC]

Hello @csuhan, Thank you very much for sharing your great work! I am wondering if ReDet is supporting inference with only CPU?

If it is supporting, how should we do it? If not, will you consider to support it in the future?

[csuhan]

ReDet supports cpu inference. Try to set device='cpu' in https://github.com/csuhan/ReDet/blob/0b9addf3c2734659fd6ffc7824f2e659fde4419c/demo_large_image.py#L91

[DangChuong-DC]

Hi @csuhan, Thank you very much for your prompt response! Running inference with GPU(s) works totally fine. However, when I tried your suggestion for running inference with cpu only, there is an error, it seems like some piece of the model is still in 'cuda:0' I provided the screen shot of the error, do you have any thoughts on this?

[jigongbao]

@csuhan i have the similar prombles too , i try to run inference with cuda:4 but there is the same error.

[DangChuong-DC]

@jigongbao your problem can be solved by changing the code in mmdet/models/anchor_head.py as below, I guess but not sure just change the device variable in anchor_generator to match with device of your input& model weights

@csuhan For my problem with CPU inference, I modified a bit and it comes to this error of those files in mmdet/models/ops since those .cpp and .cu scripts have condition of being CUDA Tensor. Do you have any thoughts to resolve this? -> I can try And thanks for your earlier suggestions

[csuhan]

@DangChuong-DC You are right. The codebase of ReDet is a rather early version of mmdet, and some ops do not suport running on cpu. I suggest to use corresponding ops in mmcv which support cpu inference. For example, RoIAlign: https://github.com/open-mmlab/mmcv/blob/master/mmcv/ops/roi_align.py RoIAlign Rotated: https://github.com/open-mmlab/mmcv/blob/master/mmcv/ops/roi_align_rotated.py

But for our RiRoIAlign, I have no time to implement the cpu version. If you still want to use it, please refer to: https://github.com/csuhan/ReDet/blob/master/mmdet/ops/riroi_align/src/riroi_align_kernel.cu https://github.com/open-mmlab/mmcv/blob/master/mmcv/ops/csrc/pytorch/cpu/roi_align_rotated.cpp I think it would not be hard to implement the cpu version.

[jigongbao]

@DangChuong-DC i tried , but there is a new error.... but thanks

[csuhan]

@jigongbao , try nms_cpu instead. https://github.com/csuhan/ReDet/blob/0b9addf3c2734659fd6ffc7824f2e659fde4419c/mmdet/ops/nms/nms_wrapper.py#L48-L51

[jigongbao]

@csuhan thanks i try to change some code in other place and it works correctly.

[DangChuong-DC]

@csuhan Thanks a lot for your explanations, I will have a look on it