FSDP + QDoRA Support

[iseesaw]

Hi the team, great work!

QDoRA seems to be better than QLoRA, refer to Efficient finetuning of Llama 3 with FSDP QDoRA

I wonder whether there will be demo / example about FSDP + QDoRA during finetuning?

Thanks!