keras-nlp
keras-nlp copied to clipboard
Cannot convert gpt2 generate into concrete function for tf lite
Describe the bug
When I want to convert a generate function to a concrete function for tf lite, it errors out with AttributeError: 'Tensor' object has no attribute 'numpy', the stack trace didn't seem to lead me to some possible cause, so want to bring it up here to see if anyone have some ideas.
To Reproduce
gpt2_preprocessor = keras_nlp.models.GPT2CausalLMPreprocessor.from_preset(
"gpt2_base_en",
sequence_length=256,
add_end_token=True,
)
gpt2_lm = keras_nlp.models.GPT2CausalLM.from_preset(
"gpt2_base_en",
preprocessor=gpt2_preprocessor)
@tf.function
def generate(prompt, max_length):
return gpt2_lm.generate(prompt, max_length)
concrete_func = generate.get_concrete_function(
tf.TensorSpec([], tf.string), 20)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
[<ipython-input-7-6300d7c3b49e>](https://localhost:8080/#) in <cell line: 7>()
5 dtype=tf.string)
6
----> 7 concrete_func = generate.get_concrete_function(
8 tf.TensorSpec([], tf.string), default_max_len)
36 frames
[/usr/local/lib/python3.10/dist-packages/tensorflow/python/framework/ops.py](https://localhost:8080/#) in __getattr__(self, name)
441 np_config.enable_numpy_behavior()
442 """)
--> 443 self.__getattribute__(name)
444
445 @staticmethod
AttributeError: in user code:
File "<ipython-input-7-6300d7c3b49e>", line 5, in generate *
dtype=tf.string)
File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/models/gpt2/gpt2_causal_lm.py", line 502, in generate *
return self._normalize_generate_outputs(outputs, input_is_scalar)
File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/models/gpt2/gpt2_causal_lm.py", line 422, in normalize *
return tensor_to_string_list(x) if is_string else x.numpy()
File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/utils/tf_utils.py", line 65, in tensor_to_string_list *
list_outputs = tensor_to_list(inputs)
File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/utils/tf_utils.py", line 50, in tensor_to_list *
list_outputs = inputs.numpy()
AttributeError: 'Tensor' object has no attribute 'numpy'
Expected behavior
Additional context
N/A
Would you like to help us fix it?
I would love to if someone can point me to more information.