keras-nlp
keras-nlp copied to clipboard
Use preallocated buffer for text generation
The current text generation implementation relies on tf.concat, which is not XLA compatible. Instead we can use a pre-allocated buffer to hold generated tokens.
As we are currently using generation util mainly for demo/guide, we can revisit this issue when we decide to bake this util into model.