ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard
verified publisher icon hpcaitech

[FEATURE]: Lora/QLora in GeminiPlugin and TorchFSDP

Open airlsyn opened this issue 1 year ago • 1 comments

Describe the feature

How can we support LoRA/QLoRA in Gemini or TorchFSDP plugin?

If there’s documentation on this feature, it might encourage community contributions.

Thanks a lot.

airlsyn avatar Nov 16 '24 14:11 airlsyn

Fond this PR https://github.com/hpcaitech/ColossalAI/pull/5001.

It’s been a long time since the last update.

airlsyn avatar Nov 18 '24 04:11 airlsyn