LLaMA-Factory icon indicating copy to clipboard operation
LLaMA-Factory copied to clipboard

Published 20 hours ago • verified publisher icon hiyouga

FSDP + Qlora Faill

Open tanghui315 opened this issue 1 month ago • 0 comments

Reminder

  • [X] I have read the README and searched the existing issues.

Reproduction

I pulled new code and ran Accelerate +FSDP + Qlora training, but encountered an error: image ValueError: Must flatten tensors with uniform dtype but got torch.bfloat16 and torch.float16

However, when I use the old code, it works fine. I suspect there is a bug in the new code.

Expected behavior

No response

System Info

No response

Others

No response

tanghui315 avatar May 22 '24 01:05 tanghui315