litgpt icon indicating copy to clipboard operation
litgpt copied to clipboard

Published 20 hours ago • verified publisher icon Lightning-AI

QLoRA with FSDP Support

Open strickvl opened this issue 11 months ago • 1 comments

I see that PEFT brought in QLoRA with FSDP support in their latest release.

Any plans to incorporate this into litgpt?

strickvl avatar Mar 21 '24 12:03 strickvl

Surely. It will need to be supported in Lightning first though. @awaelchli already had a look but there are some technical limitations to overcome

carmocca avatar Mar 24 '24 22:03 carmocca