Added EfficientNet lite option.

[sebastian-sz]

What does this PR do

This PR adds lite option to existing EfficientNet V1's.

EfficientNet lite's are slightly modified efficientnets that better support edge devices.

The models and weights are ported from original repository: https://github.com/tensorflow/tpu/blob/master/models/official/efficientnet/lite/README.md

Linked issues:

In Keras repo: #16835
Previously in Tensorflow repo: https://github.com/tensorflow/tensorflow/pull/48400 https://github.com/tensorflow/tensorflow/issues/45091

How does this affect the API

This works similar to minimalistic flag in MobileNetV3. Users can create EfficientNet lite's by passing lite=True argument upon model creation:

model = tf.keras.applications.efficientnet.EfficientNetB0(lite=True)

(Keep in mind that the above, will raise an error as the weights are not in the GCS bucket. The above works with weights=None)

NOTE: lite options have been only released for B0-B4 variants.

Pretrained weights

The weights have been converted from the original repository, using already present efficientnet_weight_update_util.py.

I cannot attach them to this PR as they exceed the allowed file size. They can be downloaded from my Google Drive: https://drive.google.com/file/d/1XSpE1NGhbaP8dN1Rem3QrmeuC--2E9QR/view?usp=sharing

Imagenet Accuracy

I evaluated the models using my external repository, so I'm copy pasting the accuracy results below:

Results table

Variant	Image size	Reported Top 1	This repo Top 1	This repo Top 5	Top 1 difference
B0	224	75.1	75.1	92.3	0.0
B1	240	76.7	76.8	93.3	+0.1
B2	260	77.6	77.6	93.8	0.0
B3	280	79.8	79.4*	94.7	-0.4
B4	300	81.5	80.8*	95.2	-0.7

We can observe that the accuracy for B3/B4 variants slightly differs from reported. The errors are below 1% and the model outputs the same probabilities as the ones in the original repository.

If this is an issue we can discuss the origin of the differences - let me know.

Known Issues

Similar to minimalistic MobileNet's V3 the models are not covered under applications_test.py, because they are created based on a parameter. It would be nice to have those tests, I can try to further modify applications_test.py to try and include these models. Let me know.

[sebastian-sz]

I might have accidentally removed the request for review from @haifeng-jin - I did not mean to do that.

[tanzhenyu]

Re Pretrained Weights -- do you have a colab that can help us evaluate the model using the weights from Google Drive?

[sebastian-sz]

@tanzhenyu Here is a Colab link

Few notes:

• The evaluation requires TFRecords created via scripts in tensorflow/tpu repository.
• The colab is mostly notebook version of eval script from my repo

Let me know if this works for you!

[tanzhenyu]

@tanzhenyu Here is a Colab link

Few notes:

• The evaluation requires TFRecords created via scripts in tensorflow/tpu repository.
• The colab is mostly notebook version of eval script from my repo

Let me know if this works for you!

Looks good to me!

[LukeWood]

@sebastian-sz can we move this PR to KerasCV?

[sebastian-sz]

@LukeWood yes, I will try to port this PR until the end of the month. I will close this PR once I port it to keras-cv (unless you want to close now).

[LukeWood]

@LukeWood yes, I will try to port this PR until the end of the month. I will close this PR once I port it to keras-cv (unless you want to close now).

Thank you! Keep me posted, feel free to request review.

[sebastian-sz]

This PR has been moved to keras-cv. Closing this for now.