Jonathan Bischof

I haven't tested if extraction is working for other formats like `.zip` or `.tar.bz`. This is probably worth looking into.

I think ideally that the method should return a path to the extracted file @Pouyanpi. In my colab we see that `gzip_file` points to the downloaded file `/content/bert_base_uncased.tar.gz`. However, if...

@Pouyanpi are you still interested in working on this bug? Please let us know and thanks for your contributions!

Duplicate of #17177, which I opened to allow for a new contributor.

Would love to see this incorporated with our new preprocessors as well (#343)!

Long term BERT should also be our first e2e pretraining example using the model API rather than a bunch of custom code.

Some thoughts: * Can you give some examples where the last layer is not used as a summary? I have not seen something like this since ELMO. It's just a...

Great work! Mostly minor changes to sync with the final version of #361. Not sure about our initializer strategy however. Will need to check up on that.

Thanks for adding the defaults! Added some small comment about adding test coverage.

@abheesht17 I have moved the gpt2 checkpoints: ``` gs://keras-nlp/models/gpt2_base/: gs://keras-nlp/models/gpt2_base/merges.txt gs://keras-nlp/models/gpt2_base/model.h5 gs://keras-nlp/models/gpt2_base/vocab.json gs://keras-nlp/models/gpt2_extra_large/: gs://keras-nlp/models/gpt2_extra_large/merges.txt gs://keras-nlp/models/gpt2_extra_large/model.h5 gs://keras-nlp/models/gpt2_extra_large/vocab.json gs://keras-nlp/models/gpt2_large/: gs://keras-nlp/models/gpt2_large/merges.txt gs://keras-nlp/models/gpt2_large/model.h5 gs://keras-nlp/models/gpt2_large/vocab.json gs://keras-nlp/models/gpt2_medium/: gs://keras-nlp/models/gpt2_medium/merges.txt gs://keras-nlp/models/gpt2_medium/model.h5 gs://keras-nlp/models/gpt2_medium/vocab.json