2rd

Results 3 comments of 2rd

[Usage]: Not enough memory when run a 33b model float16 on 2 x L40 GPU (48G)

It's for me occur when I set the gpu_memory_utilization to small rate so that the preallocation memory not enough, but actually I got the totally enough memory just not to...

[Bug]: After updating to vllm 0.5.3.post1, "Exception in worker VllmWorkerProcess while processing method init_device: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method."

set environment variable VLLM_WORKER_MULTIPROC_METHOD=spawn

【HACKATHON 7th Code Camp】黑客松护航计划集训营（正式批）

【报名】：3 【其他意向】： 不接受调剂