Truncation in Tokenizer?

[sedrick-keh-tri]

Seems like the current RWKVWorldTokenizer on HuggingFace does not do truncation, even though I set truncation=True. Is this a deliberate decision? Is there some other way to do input truncation?

[BlinkDL]

please raise the issue in https://github.com/huggingface/transformers/pull/26963 :)