keras-nlp

keras-nlp copied to clipboard

Add a byte or character level seq to seq example on keras.io

4

This could be similar to very similar in structure to the [lstm seq2seq guide](https://keras.io/examples/nlp/lstm_seq2seq/) on keras.io, but show using either the ByteTokenizer or UnicodeCharacterTokenizer (or both). We should demo training...

mattdangerw

TransformerDecoder should support single inputs

2

Currently TransformerDecoder must have both `encoder_inputs` and `decoder_inputs`, while for models like GPT2, only `decoder_inputs` is required. We should make a change to mark `encoder_inputs` as optional.

chenmoneygithub

Add a translation example on keras.io

4

We should write a example very similar to the [Spanish English translation example](https://keras.io/examples/nlp/neural_machine_translation_with_transformer/) already on keras.io. We can use the same dataset, and same basic model structure, but we should...

mattdangerw

Integrate TransformerEncoder into the BERT example

4

We should update https://github.com/keras-team/keras-nlp/tree/master/examples/bert to use `keras_nlp.layers.TransformerEncoder`.

mattdangerw

Add a keras.io example using KerasNLP!

14

KerasNLP is always looking for new examples on [keras.io](https://keras.io/keras_nlp) the demonstrate how to use the library. This issue will stay open on a "contributions welcome" list forever. If you have...

mattdangerw

Currently we are using the BackupAndRestore callback to resume training on our examples after a failure. We also need to make sure that we reset the dataset iterator to the...

mattdangerw

Integrating MLMMaskGenerator into BERT example

3

Fixes #166 Hey @chenmoneygithub Following our discussion I think the PR is ready for review!

aflah02

Add a vocabulary_size argument to WordPieceTokenizer

18

We should add a `vocabulary_size` argument to the WordPieceTokenizer layer that forces the vocabulary size by truncating the passed in vocabulary if necessary. Potential docstring: ``` vocabulary_size: Force the vocabulary...

mattdangerw

Integrate MLMMaskGenerator into BERT example

4

Currently the BERT example writes [custom code](https://github.com/keras-team/keras-nlp/blob/master/examples/bert/create_pretraining_data.py#L392) to generate MLM mask, which is slow. We should replace it with the [MLMMaskGenerator](https://github.com/keras-team/keras-nlp/blob/master/keras_nlp/layers/mlm_mask_generator.py).

chenmoneygithub

[Training] Track the progress of BERT Training

3

This is an issue for tracking the progress of training BERT example. The model has different sizes: tiny, small, base and large. Only tiny and small fit in a common...

chenmoneygithub