alignment-handbook icon indicating copy to clipboard operation
alignment-handbook copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Dependency updates for QLoRA+FSDP

Open deep-diver opened this issue 1 year ago • 0 comments

For QLoRA+FSDP support, the dependencies should be updated:

  • bitsandbytes>=0.43.0
  • accelerate>=0.28.0
  • transformers>4.38.2
  • trl>0.7.11
  • peft>0.9.0

Also, it would be wonderful to have accelerate recipe for this too.

deep-diver avatar Apr 16 '24 15:04 deep-diver