Conv2d_transpose requires asymmetric padding which the CUDA EP currently does not support

[ultimatedigiman]

Describe the bug After successfully converting a tensorflow frozen graph(.pb) into a .oonx model, I encountered a warning while inferring the onnx model with onnxruntime-gpu in python. The warning says conv2d_transpose op is not supported because it requires asymmetric padding which the CUDA EP currently does not support.

But it's strange that I found ConvTranspose is in Supported ONNX Operators and it seems the asymmetric padding issue is already solved.

System information

• OS Platform and Distribution:Linux Ubuntu 20.04
• ONNX Runtime installed from: pip install onnxruntime-gpu
• ONNX Runtime version: 1.10.0
• Python version: 3.6
• CUDA/cuDNN version: 11.3/8.2.1
• Tensorflow Version: 1.15
• opset: 13

Expected behavior conv2d_transpose has been supported.

Screenshots

Additional context I found tf.nn.conv2d_transpose has been tested in tensorflow-onnx with padding="VALID",while I use padding="SAME". https://github.com/onnx/tensorflow-onnx/blob/f64772ce166ea2a0402524d9741b2fb71e5663df/tests/test_backend.py#L579-L590

[hariharans29]

The PR you are referring to: https://github.com/microsoft/onnxruntime/pull/4627 supports asymmetric padding in CUDA Conv kernel (not ConvTranspose). We haven't seen too many models requiring asymmetric padding support for ConvTranspose in the CUDA EP and it is a backlog item. If more community members require this, it can be prioritized.

[uvbkq]

Are there any updates on this item or whether its backlog position has been changed? I believe this might be blocking for a lot of Autocoders, UNets or others requiring the model output size to match its input size when training with Tensorflow?

[OvervCW]

For us this is currently making the ONNX runtime unusable because we have a lot of segmentation models that are all taking 12x as long compared to TensorFlow/TensorRT.

[dBeker]

Is there any progress in this?

[jensk1]

Any news on this?

[ffd000]

Any updates? I think enough community members have reacted to this already.

[Toeplitz]

@hariharans29 do you have any updates on this?

This is blocking us from using some of our existing models in onnx.

2024-02-23 10:12:27.065850685 [W:onnxruntime:, cuda_execution_provider.cc:2319 ConvTransposeNeedFallbackToCPU] Dropping the ConvTranspose node: model/conv3d_transpose/conv3d_transpose to CPU because it requires asymmetric padding which the CUDA EP currently does not support