Fallqs
Results
1
comments of
Fallqs
It turn outs to be a problem with tensor parallel in hybrid_parallel plugin, where lora parameters are ignored when building column&row parallel layers.