Jason Wolosonovich

Any movement on this?

@bandsina @kristapratico do we think this will be something addressed soon or will it take some time? just curious because we're in the process of testing library upgrades on our...

@jeff-shepherd thanks so much!

@jeff-shepherd one more question. any idea on the timeline for the next release?

Perfect, thank you @jeff-shepherd!

Should also add that I'm running Win 7 (64-bit)

This solution worked like a charm! Note: The place to enter the code is highlighted in the .png I attached. In my Vagrantfile that was at line 60. The solution...

I'd like DistilBERT for both TF and PT please

Aren't the "fast" tokenizers in `transformers` these rust-based ones? The `AutoTokenizer` class uses the `PreTrainedTokenizersFast` class by default. I've run into this exact issue this week. The `AutoTokenizer` fast implementations...

Thank you for confirming @ydshieh I actually opened a feature request a couple weeks ago in [#1058 ](https://github.com/huggingface/tokenizers/issues/1058) that also demonstrates the behavior. @Narsil In terms of expected behavior, that...