neural-api icon indicating copy to clipboard operation
neural-api copied to clipboard

Implement hard sigmoid and hard swish

Open joaopauloschuler opened this issue 4 years ago • 0 comments

Use this code as inspiration: def HardSigmoid(x): return tensorflow.keras.layers.ReLU( 6.0 )( x + 3.0 ) * ( 1.0 / 6.0 )

def HardSwish(x): return tensorflow.keras.layers.Multiply()([tensorflow.keras.layers.Activation(HardSigmoid)(x), x])

joaopauloschuler avatar Oct 12 '21 09:10 joaopauloschuler