owanr

Yes! Sorry for not including that earlier. I'm trying to run the program in Python 3.6 and Pytorch 1.5 since I want to try combining it with another project which...

@zhiyixu The code being referred to should go into .../site-packages/transformers/models/auto/tokenization_auto.py However, what worked for me was updating my transformers and tokenizers package. tokenization_auto.py has a mapping of tokenizers at the...