Albert Zeyer

The initial version is done already (`export_to_onnx.py`) and works for pure PT code, so I think the main work is done. We can already close the issue. But let's anyway...

One aspect is that convolution with `padding="same"` is not supported by ONNX. (See also here: https://github.com/rwth-i6/returnn/issues/1120#issuecomment-1541471146) However, we can simply work around that by doing the padding manually. The code...

`padding="valid"` is just the same as `padding=0`, or not?

> Anyway, `padding="valid"` is the easy case to fix, since then we only have to set `padding = 0`, as you said. Yes, my question was just to confirm this....

Btw, next step after the current demo works: Test our RF Conformer implementation.

Actually, I just wonder now: Instead of manually adding the padding to the input, could we maybe just set `padding = (filter_size - 1) // 2`? But probably not really...

> I was able to insert `torch.onnx.is_in_export_mode()` into the already developed code which creates manual striding. However, this only works for the RF demo: the PT demo as it is...

> Internally, the `forward()` implementation of convolutional layers is as follows ... This is only for the case `padding_mode != 'zeros'`, which we don't support at all currently, right?

I'm not exactly sure what you mean. The padding numbers don't need to be calculated at all. `F.pad` will just add them. `F.pad` will pad the input. The input is...

> The number of operations in the RF graph is much bigger than the number of operations in the PT graph. That's very interesting. Thanks for this. We should study...