While doing tflite lite model conversion with input signature as string data type i am getting this error
ConverterError: /usr/local/lib/python3.10/dist-packages/tensorflow/python/framework/func_graph.py:670:0: error: 'tf.FastWordpieceTokenizeWithOffsets' op is neither a custom op nor a flex op
:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/usr/local/lib/python3.10/dist-packages/tensorflow/python/framework/func_graph.py:670:0: note: Error code: ERROR_NEEDS_CUSTOM_OPS
/usr/local/lib/python3.10/dist-packages/tensorflow/python/framework/func_graph.py:670:0: error: 'tf.TFText>FastWordpieceDetokenize' op is neither a custom op nor a flex op
:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/usr/local/lib/python3.10/dist-packages/tensorflow/python/framework/func_graph.py:670:0: note: Error code: ERROR_NEEDS_CUSTOM_OPS
:0: error: failed while converting: 'main':
Some ops in the model are custom ops, See instructions to implement custom ops: https://www.tensorflow.org/lite/guide/ops_custom
Custom ops: FastWordpieceTokenizeWithOffsets, TFText>FastWordpieceDetokenize
Details:
tf.FastWordpieceTokenizeWithOffsets(tensor<?x!tf_type.string>, tensor<241460xui8>) -> (tensor<?x!tf_type.string>, tensor<?xi64>, tensor<?xi64>, tensor<?xi64>, tensor<?xi64>) : {device = "/device:CPU:0"}
tf.TFText>FastWordpieceDetokenize(tensor<40xi32>, tensor<2xi64>, tensor<339052xui8>) -> (tensor<?x!tf_type.string>) : {device = ""}
kindly please check this below colab link
https://colab.research.google.com/drive/14_ajVH4r4NXN6cDw96XJNBtltoueoGfN?usp=sharing
Thanks for checking this out!
Have you tried registering the tf lite custom ops along with the other ops in your converter? That would be the first thing I try here.
Here's a guide -> https://www.tensorflow.org/text/guide/text_tf_lite
keras-team
