tokenizers icon indicating copy to clipboard operation
tokenizers copied to clipboard

`Encoding` object stub doesn't include `__len__`

Open thearchitector opened this issue 2 weeks ago • 2 comments

It looks like the Encoding object returned by Tokenizer.encode has the __len__ dunder method, and running len(encoding) works, but for some reason it's corresponding generated .pyi file does not. This causes type checking errors in Pyright/Pylance, even when running in non-strict mode (since this library isn't typed).

thearchitector avatar Jun 18 '24 15:06 thearchitector