Kevin_Xiong
Kevin_Xiong
我也有这个问题，NICE能完美跑，glow就不行，compile也能过，fit时就不行
Have you resolved this? I've been stuck here for days. @ch-tiger1
with NCCL_DEBUG_SUBSYS=ALL NCCL_DEBUG=TRACE env, I found that it turned out to be OOM when capturing the graph
Official Deepseek API  Sglang after this PR  they both return extra`\n`s to keep connection alive. ``` python from openai import OpenAI openai_api_key = "EMPTY" openai_api_base = "http://localhost:30000/v1" client...
> I am not sure if I cache your point, how will additional `\n` enhance behavior? My concern is that since `adapter` is for all LLMs and this change is...
> I see, let's say we are serving a relatively much smaller model such as Qwen-7B, will this kind of mechanism be still suitable? It would better if we can...
> I discussed this issue with [@liuhe-spec](https://github.com/liuhe-spec) on WeChat, and we strongly suspect it is likely related to RoCE network congestion control. > > If possible, you can ask your...