coremltools icon indicating copy to clipboard operation
coremltools copied to clipboard

torch.nn.functional.grid_sample for volumetric input/ Rank 5 input and grid

Open nighting0le01 opened this issue 1 year ago • 3 comments

🌱 Describe your Feature Request

support torch.nn.function.grid_sample for 5d Volumetric input by supporting 5d input for mb.affine and mb.resample for rank >=4 :https://pytorch.org/docs/stable/generated/torch.nn.functional.grid_sample.html

How can this feature be used?

Please provide some examples where this feature can be used. For video generation models features on device

Describe alternatives you've considered

currently grid_sampler is supported for rank 4 tensors. it erros out in resample op mb.resample input "x" to the "resample" op must be a rank 4 tensor. Got rank 5 tensor of shape (22, 4, c,h,w)

please support to get mb.affine and mb.resample for rank 5 inputs

nighting0le01 avatar Jul 23 '24 03:07 nighting0le01

@junpeiz @TobyRoseman @aseemw this feature is needed for video gen application

nighting0le01 avatar Jul 23 '24 04:07 nighting0le01

This sounds like a limitation in the Framework :)

jakesabathia2 avatar Jul 26 '24 16:07 jakesabathia2

hello @jakesabathia2 what are the steps to support such a custom layer? do we have a up to date example. So i can implement and land the PR

nighting0le01 avatar Jul 26 '24 17:07 nighting0le01