Is it possible to have quantised fused_tanh (with add op ) via TFlite converter

[arm-ssingh]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
• TensorFlow installed from (source or binary): Source
• TensorFlow version (or github SHA if from source): 2.6.0

Provide the text output from tflite_convert

When comparing against the quantized Keras output. The TfLite output does not match the Keras output.

Standalone code to reproduce the issue Provide a reproducible test case that is the bare minimum necessary to generate the problem. If possible, please share a link to Colab/Jupyter/any notebook.

https://colab.research.google.com/drive/1IqfO1FTKBahe9iahqD4ZgjGnkV1Utulk?usp=sharing

Also, please include a link to a GraphDef or the model if possible.

Any other info / logs

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

Is it possible to have an ADD operator with a fused TANH activation in TFL ? Such quantized int16 model is producing wrong results compare to Keras output ?
Model:
=====
input = tf.keras.layers.Input(shape=  (1, 128, 3), batch_size=1)
add = tf.keras.layers.Add()([input, input])
act = tf.keras.layers.Activation(tf.keras.activations.tanh)(add)
model = tf.keras.Model(inputs=input, outputs= act)
=======


```From the schema it seems it's supported:
`
table AddOptions
{ fused_activation_function:ActivationFunctionType; // Parameters supported by version 3. pot_scale_int16:bool = true;}

enum ActivationFunctionType : byte {
NONE = 0,
RELU = 1,
RELU_N1_TO_1 = 2,
RELU6 = 3,
TANH = 4,
SIGN_BIT = 5,
}
`

But MLIR converter currently only fuses TFL_ReluOp, TFL_Relu6Op and TFL_Relu1Op:
https://github.com/tensorflow/tensorflow/blob/77d8c333405a080c57850c45531dbbf077b2bd0e/tensorflow/compiler/mlir/lite/transforms/optimize_patterns.td#L86
https://github.com/tensorflow/tensorflow/blob/77d8c333405a080c57850c45531dbbf077b2bd0e/tensorflow/compiler/mlir/lite/transforms/optimize_patterns.td#L429

Can you confirm if it's possible to fuse 16-bit tanh in quantised model ?

[teijeong]

Hi @arm-ssingh ,

I can't reproduce colab as I don't have the input file. Meanwhile, can you confirm that the model is 16x8 quantized? Need to inspect the model, but seems it's not running quantization pass appropriately.

To do int16x8 quantization appropriately, please see the guide: https://www.tensorflow.org/lite/performance/post_training_integer_quant_16x8

[sushreebarsa]

@arm-ssingh Could you refer the comment above and let us know if it helps? Thank you!