keras-nlp icon indicating copy to clipboard operation
keras-nlp copied to clipboard

Cannot convert gpt2 generate into concrete function for tf lite

Open tianhaoz95 opened this issue 2 years ago • 4 comments

Describe the bug

When I want to convert a generate function to a concrete function for tf lite, it errors out with AttributeError: 'Tensor' object has no attribute 'numpy', the stack trace didn't seem to lead me to some possible cause, so want to bring it up here to see if anyone have some ideas.

To Reproduce

gpt2_preprocessor = keras_nlp.models.GPT2CausalLMPreprocessor.from_preset(
    "gpt2_base_en",
    sequence_length=256,
    add_end_token=True,
)

gpt2_lm = keras_nlp.models.GPT2CausalLM.from_preset(
    "gpt2_base_en",
    preprocessor=gpt2_preprocessor)

@tf.function
def generate(prompt, max_length):
    return gpt2_lm.generate(prompt, max_length)

concrete_func = generate.get_concrete_function(
    tf.TensorSpec([], tf.string), 20)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-7-6300d7c3b49e>](https://localhost:8080/#) in <cell line: 7>()
      5         dtype=tf.string)
      6 
----> 7 concrete_func = generate.get_concrete_function(
      8     tf.TensorSpec([], tf.string), default_max_len)

36 frames
[/usr/local/lib/python3.10/dist-packages/tensorflow/python/framework/ops.py](https://localhost:8080/#) in __getattr__(self, name)
    441         np_config.enable_numpy_behavior()
    442       """)
--> 443     self.__getattribute__(name)
    444 
    445   @staticmethod

AttributeError: in user code:

    File "<ipython-input-7-6300d7c3b49e>", line 5, in generate  *
        dtype=tf.string)
    File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/models/gpt2/gpt2_causal_lm.py", line 502, in generate  *
        return self._normalize_generate_outputs(outputs, input_is_scalar)
    File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/models/gpt2/gpt2_causal_lm.py", line 422, in normalize  *
        return tensor_to_string_list(x) if is_string else x.numpy()
    File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/utils/tf_utils.py", line 65, in tensor_to_string_list  *
        list_outputs = tensor_to_list(inputs)
    File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/utils/tf_utils.py", line 50, in tensor_to_list  *
        list_outputs = inputs.numpy()

    AttributeError: 'Tensor' object has no attribute 'numpy'

Expected behavior

Additional context

N/A

Would you like to help us fix it?

I would love to if someone can point me to more information.

tianhaoz95 avatar Jun 25 '23 02:06 tianhaoz95