keras-nlp
keras-nlp copied to clipboard
keras-team
Add Function(s)/Class(es) for Decoding Strategies [Text Generation]
Not sure if this is worth implementing, but basically, we can have functions/classes, which users can call in order to get decoded text. As far as I know, we have these decoding strategies:
- Greedy Sampling
- Random Sampling
- Top-k Sampling
- Top-p (Nucleus Sampling)
- Beam Search
Users can feed a tensor of size (bsz, seq_len, vocab_size) (output of a generation model), and choose a decoding strategy from above. The output will be the decoded tokens (of size (bsz, seq_len)).
I realised this while implementing text generation metrics like ROUGE (https://github.com/keras-team/keras-nlp/issues/38). In metrics like ROUGE, we can provide an option to the user decoding_strategy, which can take values None, "greedy", "top_k", "nucleus", etc. and decode the output accordingly. In case of None, the user will be asked to provide a decoded output.
Something like this:
from keras_nlp.utils.text_decoder import TextDecoder
class Rouge:
def __init__(self, decoding_strategy, ...):
self.decoding_strategy = decoding_strategy
if decoding_strategy:
self.text_decoder = TextDecoder(decoding_strategy)
def update_state(self, y_true, y_pred):
if self.decoding_strategy:
y_pred_tokens = self.text_decoder(y_pred)
else:
assert tf.rank(y_pred) == 2, "Please provide the decoded output, or set the decoding_strategy to a non-None value."
...
def result(...):
...
def reset_state(...):
...
@abheesht17 Thanks for opening this feature request!
Yes, autoregressive generation is something we are looking to add. We will need some discussions on which technique we want to support (greedy search is easier to implement, but beam search is better), please stay tuned, thanks!
Hi @chenmoneygithub, I can provide implementations for greedy search and beam search if you are interested.
@monatis Sorry, somehow I missed your message...
We are actively working on determining the input/output format and in which form text decoding is supported. Once we are ready, we will open up issues for contribution with a high-level guidance. Thanks for being willing to contribute!