Add BPE Text Encoder

Open PetrochukM opened this issue 7 years ago • 3 comments

Tokenizer Reference: https://github.com/eladhoffer/seq2seq.pytorch/blob/master/seq2seq/tools/tokenizer.py

Mar 25 '18 19:03 PetrochukM

No, it's an open issue. Would you like to add it?

Jul 18 '19 04:07 PetrochukM

hi, my co-worker and I finish this part of work using subword-nmt lib recently. We are looking forwards to your review & advice.

May 15 '20 05:05 Columbine21

this is issue is still open?

Aug 06 '22 13:08 abhishinde24