Bounding box augmentation with Keras ImageDataGenerator

[ohadlights]

Link to question asked in stackoverflow.

The issue that I raised is the missing option to adjust labels like bounding boxes and key-points after augmenting an image with ImageDataGenerator.

For example, the applied transformation matrix is not returned to the user.

As far as I know, the status today is that if a user is training a bounding box detector, he can't use the ImageDataGenerator.

There are several ways to deal with this issue. For example, the ImageDataGenerator can add a callback that will allow to update the labels. Another option is to return the transformation matrix. Or probably best, the ImageDataGenerator can take a list of points and adjust them according to the transformation that is applied on each image.

[Alex1101a]

is there now a solution to this problem/question? I am also interested in this topic..

[mtourne]

This is interesting to me as well, I think for now we have to roll our own image augmentation that computes the resulting bounding box ?

[rabitwhte]

Any news on the topic?

[Dref360]

Following this RFC, I think we should make Keras preprocessing layers for this task. (Inside TF Repo)