Add support for padding and cropping to tf.keras.layers.experimental.preprocessing.Resizing

[willbattel]

Please make sure that this is a feature request. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template

System information

• TensorFlow version (you are using): 2.4.0
• Are you willing to contribute it (Yes/No): No, unfortunately I don't have the time to do so.

Describe the feature and the current behavior/state. The new layer tf.keras.layers.experimental.preprocessing.Resizing allows for models to be made more portable by handling image resizing within the model itself, as described in the docs here. This layer provides options for interpolation, but cannot be configured to crop or pad the image to maintain aspect-ratio.

In my case, I use the tf.image.resize_with_pad in my tf.data pipeline in order to maintain aspect ratio and letter-box the shorter sides. This cannot be done inside of the model without a custom or Lambda layer.

The layer should ideally also include the functionality from tf.image.resize_with_crop_or_pad.

Will this change the current api? How? Yes, but purely additive. I suggest adding an additional parameter resize_type to the layer and simply have the default value perform the current behavior. This would not affect current users, but offer the feature to those who chose to enable it.

Who will benefit with this feature? This would be handy for developers/researchers who want a simple, built-in mechanism for resizing images inside of a model but don't want the current stretching behavior. In particular, this would benefit the portability of models, where we can do more within the model itself.

Any Other info. N/A

[anilkumarKanasani]

@willbattel , we would like to work on this feature improvement and give the updated code

[gadagashwini]

@willbattel, Latest Tensorflow v2.8, tf.keras.layers.experimental.preprocessing.Resizing has crop_to_aspect_ratio

tf.keras.layers.Resizing(
    height,
    width,
    interpolation='bilinear',
    crop_to_aspect_ratio=False,
    **kwargs
)

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[willbattel]

@gadagashwini thank you for letting me know. That is great to see.

The other thing we are still interested is a pad_to_aspect_ratio option so that we can include the whole image, without cropping, in its original aspect ratio by adding padding on the short sides- the same way tf.image.resize_with_pad does.

[Venkat6871]

Hi,

Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.

The Tensorflow team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.

Please follow the release notes to stay up to date with the latest developments which are happening in the Tensorflow space.

[github-actions[bot]]

This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further.