keras-io
keras-io copied to clipboard
keras-team
New Code Example: Pre-training BERT on NSP+MLM using Hugging Face Transformers
This code example shows how to pre-train BERT using Hugging Face transformers on NSP + MLM task. Additionally, it also shows best practices for Hugging Face Datasets.
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).
View this failed invocation of the CLA check for more information.
For the most up to date status, view the checks section at the bottom of the pull request.
![google-cla[bot] avatar](https://avatars.githubusercontent.com/in/42202?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hi @fchollet,
This is part of my GSoC 2022 contribution. Hope you can review it and comment!
Thank You so much!
Thanks for the PR!
This is part of my GSoC 2022 contribution
Cool, can you provide details? Who are you working with?
Thank You for the reply. My mentor is @vasudevgupta7
I don't have any more teammates in my project and I am the solo contributor.
Hi @fchollet ,
Thank You so much for the reply and initial comments! Really appreciate it and I will incorporate the changes as suggested.
In general there is rather little Keras functionality at play here. Could we modify the example so it works better as a Keras tutorial / showcase?
I am actually a bit new to Keras/Tensorflow and was primarily following this notebook (the formatting and structure perspectives) for my tasks (my task is to create NLP examples for Keras using Hugging Face, you can follow this link and this link to get an idea of my project). Could you, if possible, suggest something that would help me satisfy the needs better for this?
Thank You again and hope you also find some time to review my other PRs ( #967 and #968 ).
Hi @fchollet ,
I think you can merge this now! I was also hoping if you could review my other PRs! Thank You again for your time!
cla/google Failing after 2s — ❌ Missing CLA from one or more contributors
Please sign the CLA!
Sorry for the delay in reviewing -- we need more folks than just me looking at these PRs.
cla/google Failing after 2s — ❌ Missing CLA from one or more contributors
Please sign the CLA!
Sorry for the delay in reviewing -- we need more folks than just me looking at these PRs.
@fchollet, this PR messed up a bit with a push from my teammate's usernames due to shared machines and google needs CLA from him too! All ready to merge in #1044 . Thank You! I will close this after that is merged!