notebooks
notebooks copied to clipboard
Error in "Why are fast tokenizers called fast" notebook
trafficstars
In this notebook, the slow tokenizer is never actually used as the mapping function uses the fast_tokenizer.
def tokenize_with_slow(examples):
return fast_tokenizer(