DeepSeek-VL How to finetune and serve the finetuned model using VLLM like tools

How to finetune and serve the finetuned model using VLLM like tools

Open monksgoyal opened this issue 1 year ago • 3 comments

trafficstars

Hi I want to finetune it on my custom OCR tasks, can you share finetuning script and also help in telling the best way to serve it very very fast.

Mar 11 '24 07:03 monksgoyal

同问

Mar 11 '24 13:03 xs818818

SGlang support would be excellent, as it doesn't seem to work out of the box.

Mar 11 '24 18:03 brucethemoose

Made an issue for it here as well: https://github.com/sgl-project/sglang/issues/297

SGlang already supports llava, so perhaps adding deepseek would not be as huge of a jump as vllm (which currently does not support vision models).

Mar 14 '24 02:03 brucethemoose