RWKV-LM-LoRA icon indicating copy to clipboard operation
RWKV-LM-LoRA copied to clipboard

Published 20 hours ago • verified publisher icon Blealtan

PSA: Issue with Multi-GPU & CUDA 12.0

Open PicoCreator opened this issue 11 months ago • 0 comments

Currently with RWKV and DeepSpeed, there seems to be an issue where it "hangs" when activating DeepSpeed with bf16

Specifically around this line Screenshot 2023-08-04 at 9 30 34 AM

Currently this is tested to be resolved in Cuda 12.2

PicoCreator avatar Aug 04 '23 01:08 PicoCreator