Chen Qian
Chen Qian
As pointed out by our user, the Tensorboard callback is doing blocking I/O, which means the training is halted until the writing finishes. This creates a performance bottleneck especially when...
`char_to_token` is a method that converts the character index to the token index. See the HuggingFace method [here](https://huggingface.co/transformers/v3.2.0/main_classes/tokenizer.html#transformers.BatchEncoding.char_to_token) This is useful in span classification tasks, such as SQuaD, as we...
From a high level it is just a classification task, but there are some details to handle. The whole workflow can be described as: - Data 1. We can use...
Currently TransformerDecoder must have both `encoder_inputs` and `decoder_inputs`, while for models like GPT2, only `decoder_inputs` is required. We should make a change to mark `encoder_inputs` as optional.
Currently the BERT example writes [custom code](https://github.com/keras-team/keras-nlp/blob/master/examples/bert/create_pretraining_data.py#L392) to generate MLM mask, which is slow. We should replace it with the [MLMMaskGenerator](https://github.com/keras-team/keras-nlp/blob/master/keras_nlp/layers/mlm_mask_generator.py).
This is an issue for tracking the progress of training BERT example. The model has different sizes: tiny, small, base and large. Only tiny and small fit in a common...
The current text generation implementation relies on tf.concat, which is not XLA compatible. Instead we can use a pre-allocated buffer to hold generated tokens. As we are currently using generation...
This is not necessarily a bug, but I find it confusing. I tried to tokenize a sequence like `"[start] have a nice day"`, but it appears that with the default...
As title suggests.