Ning Ren

icon indicating a website link https://ningcode.com

icon location California icon location ex-Googler, ex-PonyAI; Vibe Coder, Vibe Trader; Gamer & Lover

Results 3 issues of Ning Ren

fix error and support Phi-3 models

1 comment

## Why are these changes needed? Model worker (also vllm_worker) has error loading [Phi-3 models](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) "microsoft/Phi-3-mini-128k-instruct" etc.. ![Screenshot from 2024-05-07 00-58-23](https://github.com/lm-sys/FastChat/assets/4597657/17dda665-2f2e-4a38-a0c7-c64e839d8cfc) ## Checks - [x] I've run `format.sh` to lint...

[Feature Request] generate a readme doc for the backend

[Bug]: Gemini 3.0 requires a thought_signature for tool calls

1 comment

### What happened? Gemini 3.0 requires a thought_signature for tool calls, but litellm doesn't preserve this signature in streaming mode, causing errors on subsequent turns. ### Relevant log output ```shell...

bug