BaoRong comments

Results 9 comments of


                                            BaoRong

Is there a way to terminate vllm.LLM and release the GPU memory

> In the latest version of vLLM `destroy_model_parallel` has moved to `vllm.distributed.parallel_state`. The objects you have to delete have also changed: > > ```python > from vllm.distributed.parallel_state import destroy_model_parallel >...

TProcess.NLTKSentTokenizer

urllib.error.URLError:

[BUG] Hybrid Engine with DeepSpeed Stage 3 results and Llama V2 results in gibberish outputs

I made some changes to the code to make the lm_head and token_emb weight the right shape, but the hybrid engine still generate gibberish words. ~~~python def main(): accelerator =...

what devices are supported?

same question! I's trying to use megablocks in H20, but got the same error. I will try the command pip install megablocks[all].

add SGLang as rollout engine to verl

> @ocss884 @PeterSH6 As for now, people from qwen team is reporting issues on multi-nodes. We will merge single node SGLang into veRL ASAP and work with @Qiaolin-Yu on multi-node...

[Core] Memory leakage with remote() calls

mark👀

[QUESTION] Why flux gemm_rs is not faster than torch?

mark👀

[mcore] verl+megatron development tracking

mark👀

[Feature request] Integrate DeepGEMM

mark👀