T5 Tokenizer

[mcmonkey4eva]

Feature Idea

Currently the tokenizer tools only support CLIP, but not yet T5-XXL (as used in SD3 & Flux).

This will require investigating how the T5 tokenizer works inside and porting it properly (it's more complex than clip i think)

Other

No response