LoRA icon indicating copy to clipboard operation
LoRA copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Does it support model parallel? How to convert ColumnParallelLinear to lora linear?

Open LiuShixing opened this issue 1 year ago • 4 comments

how to convert Megatron-DeepSpeed ColumnParallelLinear and RowParallelLinear to lora linear layer?

ColumnParallelLinear defined in: https://github.com/microsoft/Megatron-DeepSpeed/blob/main/megatron/mpu/layers.py#L206

LiuShixing avatar Mar 10 '23 10:03 LiuShixing