tokenizers
tokenizers copied to clipboard
`Encoding` object stub doesn't include `__len__`
It looks like the Encoding
object returned by Tokenizer.encode
has the __len__
dunder method, and running len(encoding)
works, but for some reason it's corresponding generated .pyi
file does not. This causes type checking errors in Pyright/Pylance, even when running in non-strict mode (since this library isn't typed).