Porting KerasAug into KerasCV

[james77777778]

Hi KerasCV team,

I have worked on my own project KerasAug for a while to provide the powerful, performant and bug-free preprocessing/augmentation layers.

As I have noticed that keras-core is planning to refactor the preprocessing layers by incorporating TFDataLayer , it presents a opportunity to port some of the layers from KerasAug into KerasCV.

You can visit benchmarks to see the improvement by KerasAug

Moreover, KerasAug addresses a lot of existing issues within KerasCV:

• #1801
• #1784
• #1783
• #1782
• #1768
• #1557
• #1415

Will this project benefit the community? I'm willing to contribute!

[jbischof]

@james77777778 thanks for the offer. Can you explain the source of the speedups?

[james77777778]

@james77777778 thanks for the offer. Can you explain the source of the speedups?

@jbischof Sure! I pick some layers for clarification:

Layer	Why	Speedup
RandomCropAndResize	utilizes tf.image.crop_and_resize with vectorized design	+1150%
Resize	utilizes tf.image.resize with vectorized design	+1017%
RandomColorJitter	instead of dircetly using 4 preprocessing layers, implements the logic in one place to reduce the overhead	+237%
RandomGaussianBlur	utilizes tf.vectorized_map instead of tf.map_fn	+1055%
GridMask	computes random_transform in vectorized manner	+368%

Additionally:

• adds the bounding_boxes support for Resizing layer with all 3 modes (crop, pad and distort)
• gauratees that the tensor shape is static if it should be. I believe KerasCV has some bugs in Resizing as reported in #1894
• fully supports mixed_precision policy (mixed_float16 and mixed_bfloat16). Some casting needs to be added for this functionality.

If these improvements are in the plan of KerasCV, I can send the PR (maybe one fix/improvement one small PR?).

[jbischof]

Sounds great @james77777778! I would suggest starting with one or two layers with big improvements so that we can debug any issues early in the process.