MYY

icon company Northeast University icon location Shenyang, China icon location A PhD student at Northeastern University(China)

Results 14 comments of MYY

Support FSDP worker and vLLM Ascend

@as12138 Hello. Thanks for your efforts! Can this PR be directly implemented on 910b2c 64GB now?

Support FSDP worker and vLLM Ascend

> > @as12138 Hello. Thanks for your efforts! Can this PR be directly implemented on 910b2c 64GB now? > > I tested it on the Atlas 800T A2 64G ASCEND+ARM,...

[Bug]: I want to integrate vllm into LLaMA-Factory, a transformers-based LLM training framework. However, I encountered two bugs: RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method & RuntimeError: NCCL error: invalid usage (run with NCCL_DEBUG=WARN for details)

> Can you try running vLLM with `VLLM_WORKER_MULTIPROC_METHOD=spawn` set? I tried this, but got a NCCL bug: NCCL bug ``` [rank1]: Traceback (most recent call last): [rank1]: File "/localnvme/application/sc_new/myy_world_consistency/LLaMA-Factory-0.9.0-9.27main/src/llamafactory/launcher.py", line...

[Bug]: I want to integrate vllm into LLaMA-Factory, a transformers-based LLM training framework. However, I encountered two bugs: RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method & RuntimeError: NCCL error: invalid usage (run with NCCL_DEBUG=WARN for details)

> export VLLM_WORKER_MULTIPROC_METHOD=spawn I tried vllm 0.6.3.post2.dev12+g1ffc8a73 with the test demo below, however, there is still a ```spawn``` error. demo code: ``` python import os os.environ["CUDA_VISIBLE_DEVICES"] = "1,2" from transformers.utils...