david-vectorflow

[BUG] Slow Tokenizer Message is printing when the Fast Tokenizer may be in use

1

**Issue description:** For Llava 1.5 13b, if you run it with the `--tokenizer_mode "auto"` flag set, it still prints a message that the slow tokenizer is being used. Llava has...

Import Errors occurring even when dependencies are installed

2

I am unable to create a Runtime with sglang as follows `runtime = sgl.Runtime(model_path=MODEL_DIR, tokenizer_path=MODEL_DIR)`. It throws the error below: ```python ImportError: cannot import name '_set_default_torch_dtype' from 'vllm.model_executor.model_loader' (/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/__init__.py) ```...