Support for GridSample Operation

[eewindfly]

New Operator

GridSample

Describe the operator

GridSample is a widely used operator in PyTorch for spatial transformation and grid sampling tasks. It allows sampling and interpolation of grid points, which is essential for operations such as spatial transformers, image warping, and more.

Adding support for GridSample in tf2onnx would enable seamless conversion of PyTorch models that rely on this functionality, particularly for vision-related tasks.

Do you know this operator can be constructed using existing ONNX operators?

Yes, it appears that TensorFlow Graphics provides a similar operation via tfg.image.transformer.sample. It might be possible to map GridSample to this operation during the conversion process.

Is this operator used by any model currently? Which one?

Yes, GridSample is commonly used in many vision models, including: • Spatial Transformer Networks • GANs that perform image alignment or transformation • Optical flow estimation models

Notes

Adding this operator would enhance the compatibility of PyTorch models with TensorFlow/ONNX ecosystems. TensorFlow Graphics documentation: tfg.image.transformer.sample