DiegoD94

Results 14 comments of DiegoD94

Adding Share Expert Fusion for DeepSeek

> I think we should prioritize getting this in for deepseek and llama4, it seems to be a pretty clear win > Running into an issue trying it on DeepSeekV2....

Adding Share Expert Fusion for DeepSeek

> Running into an issue trying it on DeepSeekV2. @DiegoD94 could you take a look? > > ``` > VLLM_USE_V1=0 VLLM_SHARED_EXPERT_FUSION_REPLICAS=1 vllm serve deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct --tensor-parallel-size 2 --trust-remote-code --max-model-len 16384 >...

Adding Share Expert Fusion for DeepSeek

> Can you merge from main? It should fix a bunch of failed tests Hi Thanks, I still got some failed test, but majority of them are timeout error(all 4...

`asyncio.get_event_loop` raises error instead of spawning new loop when no loop is running.

Same here, multiple project, including sglang and verl is impacted