keras-team
Integrating MLMMaskGenerator into BERT example
Fixes #166 Hey @chenmoneygithub Following our discussion I think the PR is ready for review!
@aflah02 Thanks for the PR! I took a brief look and I am not sure if this works. Last time I checked the example code is working at string space, while the MLMMaskGenerator is working at integer space, so simply replacing the function call might not work.
Could you try running the script and paste the generated data preview under this PR? You can refer to the "Quickly test out the code" section, and use this command to preview the data:
python -c "from keras_nlp.utils.tensor_utils import preview_tfrecord; preview_tfrecord('/path/to/tfrecord_file')"
@aflah02 Thanks for the PR! I took a brief look and I am not sure if this works. Last time I checked the example code is working at string space, while the MLMMaskGenerator is working at integer space, so simply replacing the function call might not work.
Could you try running the script and paste the generated data preview under this PR? You can refer to the "Quickly test out the code" section, and use this command to preview the data:
python -c "from keras_nlp.utils.tensor_utils import preview_tfrecord; preview_tfrecord('/path/to/tfrecord_file')"
@chenmoneygithub Got it! I'll rework on this and test as you said! Thanks for the review!!