Instantiating a `tf.keras.applications` Model Nullifies Augmentation Layers

[beneyal]

trafficstars

Hi! 👋

I posed this issue over at TensorFlow's GitHub, and was told I should be posting this issue here.

I'm having a very weird problem which I couldn't find in the Issues tab, so I hope I'm not duplicating anything.

First, I'm using TensorFlow v2.8.0 on Google Colab (Python v3.7.12).

When creating an augmentation layer, be it a single layer, or a tf.keras.Sequential collection of them, they work as expected. However, right after instantiating a model from tf.keras.applications (I've seen it happen with EfficientNetB0 and ResNet152V2), the augmentation layers become a no-op. This behavior does not occur with TensorFlow v2.7.0 on Colab.

A Colab notebook showing this behavior can be found here: https://colab.research.google.com/drive/14K2-OgcPjkHRk2aeOMnW1CnaThow6Fia?usp=sharing

Thank you for your time and patience, Ben

[mertorelio]

This works for me augmented_img = data_augmentation(img, training=True)

[sachinprasadhs]

By default keras layers augmentation works only during training. Augmentation layers which you are using such as RandomRotation, RandomFlip, RandomZoom, RandomHeight, RandomWidth by default does augmentation only during training and remains inactive during inference.

In order to make these layers active during inference time, call the layer with training = True. Here is the detail documentation for RandomFlip.

[beneyal]

Thanks for the info and the solution! 🙂

But why does this behavior happen when instantiating a tf.keras.applications model? Does the instantiation turn on some inference flag? Because, as the notebook shows, the only difference between the "working" and "not working" cells are the instantiation of tf.keras.applications.EfficientNetB0. The data_augmentation variable is unchanged.

[fchollet]

The expected behavior is that the default training value received by the layers should be True since it defaults to True in the layer signatures and isn't set anywhere else. We should investigate why that doesn't happen after instantiating an application.

We recommend always setting training explicitly when calling data_augmentation so that there is no confusion as to wht the value used actually is: augmented_img = data_augmentation(img, training=True)

[beneyal]

Thank you, much appreciated!

Should I keep the issue open for the time being?

[fchollet]

Yes, let's keep it open until resolved.

[tilakrayal]

@beneyal, I tried to execute the mentioned code on tensorflow v2.13 twice on the coab, and observed that the original image and the augmented image both were different. Kindly find the gist of it here and let us know if you are looking for the same. https://colab.research.google.com/gist/tilakrayal/e1a8b755d16193876d98b86bea32e79c/aug_not_working_after_creating_application-1.ipynb Thank you!

[beneyal]

Looks good to me 😃 Thanks! I think this can be closed now.

[tilakrayal]

@beneyal, Thank you for the confirmation. As mentioned above, the issue got resolved in the latest version. Moving this issue to the closed status. Thank you!