neural-api
neural-api copied to clipboard
Implement hard sigmoid and hard swish
Use this code as inspiration: def HardSigmoid(x): return tensorflow.keras.layers.ReLU( 6.0 )( x + 3.0 ) * ( 1.0 / 6.0 )
def HardSwish(x): return tensorflow.keras.layers.Multiply()([tensorflow.keras.layers.Activation(HardSigmoid)(x), x])