BPE tokenizer

Open chenmoneygithub opened this issue 3 years ago • 2 comments

This PR is a rework on https://github.com/keras-team/keras-nlp/pull/303.

Recreate the PR instead of direct editing for clear remote-local tracking.

Oct 14 '22 18:10 chenmoneygithub

Thanks! Will take a look!

One note, it might be nice to add Jesse as a co-author on the commit, he did some incredible work on this and we should make sure to credit it.

https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/creating-a-commit-with-multiple-authors#creating-co-authored-commits-on-the-command-line

Oct 14 '22 20:10 mattdangerw

@mattdangerw Definitely! Will add in the next commit.

Oct 14 '22 20:10 chenmoneygithub