keras icon indicating copy to clipboard operation
keras copied to clipboard

[Bounding Boxes] Add Support for Bounding Box Transformations during Image Resizing

Open sineeli opened this issue 1 year ago • 2 comments

This PR covers below cases:

  1. Add bounding box transformations: Here we require orig_height and orig_width for bounding boxes transformation as each image can have separate bounding boxes relative to the original image size. Later we can use the same to transform bounding boxes as per the ratios.
  2. Make convert_format to be compatible in tf.data pipeline regardless of backend.

Just a temporary file without disturbing previous implementation of convert_format

sineeli avatar Oct 17 '24 06:10 sineeli

Sample notebook of using convert_format and rezie layer https://colab.research.google.com/gist/sineeli/257e4d546ed89ff64c0550712287b208/-keras3-resizing.ipynb

sineeli avatar Oct 17 '24 06:10 sineeli

Codecov Report

Attention: Patch coverage is 20.81081% with 293 lines in your changes missing coverage. Please review.

Project coverage is 73.30%. Comparing base (3d32481) to head (c5457af). Report is 8 commits behind head on master.

Files with missing lines Patch % Lines
...image_preprocessing/bounding_boxes/bounding_box.py 14.12% 220 Missing and 5 partials :warning:
...g/image_preprocessing/bounding_boxes/converters.py 34.09% 28 Missing and 1 partial :warning:
...yers/preprocessing/image_preprocessing/resizing.py 36.66% 16 Missing and 3 partials :warning:
...g/image_preprocessing/bounding_boxes/validation.py 14.28% 12 Missing :warning:
...i/_tf_keras/keras/utils/bounding_boxes/__init__.py 0.00% 5 Missing :warning:
...eprocessing/image_preprocessing/random_rotation.py 33.33% 2 Missing :warning:
keras/api/_tf_keras/keras/utils/__init__.py 0.00% 1 Missing :warning:

:exclamation: There is a different number of reports uploaded between BASE (3d32481) and HEAD (c5457af). Click for more details.

HEAD has 2 uploads less than BASE
Flag BASE (3d32481) HEAD (c5457af)
keras 4 3
keras-torch 1 0
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #20368      +/-   ##
==========================================
- Coverage   78.87%   73.30%   -5.58%     
==========================================
  Files         512      516       +4     
  Lines       49266    49563     +297     
  Branches     7953     7962       +9     
==========================================
- Hits        38861    36333    -2528     
- Misses       8546    11420    +2874     
+ Partials     1859     1810      -49     
Flag Coverage Δ
keras 73.24% <20.81%> (-5.51%) :arrow_down:
keras-jax 62.34% <20.81%> (-0.07%) :arrow_down:
keras-numpy 57.42% <20.81%> (-0.01%) :arrow_down:
keras-tensorflow 63.56% <20.81%> (-0.08%) :arrow_down:
keras-torch ?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Oct 18 '24 21:10 codecov-commenter

Reference Colab: https://colab.research.google.com/gist/sineeli/77cdc067d0adcef0a267b00b4921e7b8/-keras3-resizing.ipynb

sineeli avatar Oct 22 '24 09:10 sineeli

Can you take a look at the test failures? I think they are likely related to the fact that torch tests are running with data_format="channels_first". https://github.com/keras-team/keras/actions/runs/11469236802/job/31916066253?pr=20368

fchollet avatar Oct 22 '24 23:10 fchollet

Can you take a look at the test failures? I think they are likely related to the fact that torch tests are running with data_format="channels_first". https://github.com/keras-team/keras/actions/runs/11469236802/job/31916066253?pr=20368

Oh yes!! thanks I updated the test cases for channels_first case.

Reference:

  1. Bounding Boxes with pad_to_aspect_ratio=True
  2. Bounding Boxes with crop_to_aspect_ratio=True
  3. Normal Resize

sineeli avatar Oct 23 '24 04:10 sineeli

The code looks good to me! Not sure about the max bbox layer.

A side question is whether we need "bbox" in the name of the transformations like affine_transform etc et distinguish them from keras.ops.image.affine_transform and so on. No strong opinion.

I guess it should be fine as user will use keras.utils.bounding_boxes.affine_transform and it is explicit that its under bounding_boxes.

sineeli avatar Oct 23 '24 05:10 sineeli