shangzy comments

Repositories
Issues
Comments

Results 2 comments of


                                            shangzy

FSDP QDoRa

> Simply add `--use_dora True` to this script https://github.com/hiyouga/LLaMA-Factory/blob/main/examples/extras/fsdp_qlora/sft.sh hi, can QDora work with `fsdp_offload_params`. When try FSDP + QDora + Offload with Qwen 1.5 72B: `ValueError: Expected a cuda...

FSDP QDoRa

> I got the same error when I tried `--use_dora True`. https://github.com/huggingface/peft/pull/1724