Deepspeed support finetune extra model with lora ?

[wanghongqu]

Deepspeed support finetune extra model with lora ?

[Alli-Ekundayo]

Yes, DeepSpeed supports fine-tuning extra-large models like LoRA efficiently. DeepSpeed provides various optimizations for training and inference, including memory optimization, distributed training, and mixed precision training, which can be beneficial when fine-tuning large models like LoRA. You can leverage DeepSpeed's capabilities to accelerate the fine-tuning process and manage memory effectively.