vllm
vllm copied to clipboard
Support custom tokenizer
We should provide a clean abstraction and interface so that users can use their custom tokenizer very easily.