MobilenetV3 hard_sigmoid could be reimplemented using only keras.layers.

[maciej3031]

trafficstars

System information.

TensorFlow version: 2.9 Are you willing to contribute it (Yes/No) : Yes

Describe the feature and the current behavior/state.

Current hard_sigmoid implementation: https://github.com/keras-team/keras/blob/dee67880be30eb1a5c2dfc7b4e60b32cd4789fbe/keras/applications/mobilenet_v3.py#L559 directly uses tensorflow + and * ops. Perhaps it would be better to rewrite it in order to only use keras.layers API. It would be cleaner to have only keras.layers.Layer subclasses in the model and should simplify tasks like models rebuilding, nested models flattening etc.

Will this change the current api? How? No

Who will benefit from this feature? Everyone who rebuilds model and does not want to create workaround to deal with both tf ops and Layer subclasses. It could probably be easily reimplemented as follows:

def hard_sigmoid(x):
    x = Rescaling(scale=1.0, offset=3.0)(x)
    x = layers.ReLU(6.)(x)
    x = Rescaling(scale=1.0/6.0, offset=0.0)(x)
    return  x

[tilakrayal]

@maciej3031, Can you please share a reproducible code that supports your statement so that the issue can be easily understood? Thank you!

[maciej3031]

When you want to rebuild any layer from the model from scratch i.e. the 4-th layer wich is keras.layers.core.tf_op_layer.TFOpLambda then you could want to do this like this:

from keras.applications.mobilenet_v3 import MobileNetV3Small
mnet = MobileNetV3Small()
mnet.layers[4].__class__(**mnet.layers[4].get_config())

However, you cannot do this wit keras.layers.core.tf_op_layer.TFOpLambda and you'll get an error. If you i.e. replace:

def hard_sigmoid(x):
      return layers.ReLU(6.0)(x + 3.0) * (1.0 / 6.0)

with

def hard_sigmoid(x):
      x = layers.Rescaling(scale=1.0, offset=3.0)(x)
      x = layers.ReLU(6.)(x)
      x = layers.Rescaling(scale=1.0/6.0, offset=0.0)(x)
    return  x

Then it will replace + and * with Rescaling layer and above rebuilding code will work fine as the keras.layers.core.tf_op_layer.TFOpLambda will be replaced with keras.layers.preprocessing.image_preprocessing.Rescaling.

[haifeng-jin]

@maciej3031 Thanks for the issue! Would you like to start a PR to make the change? We will asl ensuring that the new implemenation can save and load successfully (not trigger any bug).