BiRefNet icon indicating copy to clipboard operation
BiRefNet copied to clipboard

Request: Support for deform_conv2d from an alternate repository to enable better ONNX export with dynamic_shape

Open alvarofsan opened this issue 1 year ago • 3 comments

Hi,

I was wondering if it would be possible to modify the current implementation of the model to use the DeformConv2dFunction function from MMCV instead of the official implementation from Torchvision.

The reason for this request is that the developers of MMCV have also implemented support for exporting this operator to ONNX, along with its associated ONNX Runtime operator. This would enable exporting the model to the ONNX format without relying on the current solution, deform_conv2d_onnx_exporter, which is quite slow, and would help address potential issues related to dynamic_shape handling.

Would you consider integrating or optionally supporting this alternative implementation? If needed, I’d be happy to contribute. I understand that a change of this type might require retraining the network.

Thank you for your time and consideration!

Best regards, Álvaro.

alvarofsan avatar Nov 20 '24 08:11 alvarofsan

Hi,

First, sorry for the delayed reply. I checked your issue and was thinking of it. But after a while, I was interrupted and forgot to reply to you... To be honest, I've also checked the MMDeploy before. But I didn't manage to do it. It could work, but I just don't have too much time for it. But if you find a way to transform one model, I'm very willing to transform the other ones and share them, as well as with notebooks of guidance to help others use. Overall, if you can do it, I would be very very happy!

Best Regards Peng zheng

ZhengPeng7 avatar Nov 26 '24 10:11 ZhengPeng7

Thank you for your response!

I believe transforming the model should be relatively straightforward. It would involve replacing the torchvision function with the corresponding MMCV implementation (if I’m understanding correctly) and adjusting the parameters to achieve the same behavior. However, I currently lack the resources to retrain the model from scratch.

Best regards, Álvaro :)

alvarofsan avatar Nov 26 '24 11:11 alvarofsan

Yeah, one reason I didn't use it is because the cost of re-training with MMCV for a trial (not 100% success) is too high and makes no sense. If this is necessary, the framework is too closed...

ZhengPeng7 avatar Nov 26 '24 14:11 ZhengPeng7