Ruoyu Qin

icon indicating an email [email protected]

icon company Tsinghua University icon location China icon location Ph.D. student in MADSys @ Tsinghua University. Machine Learning System. Intern at Moonshot AI.

Results 1 issues of Ruoyu Qin

[Model Loading] Speedup model loading with distributed loading

Hello! The current method for model loading is quite fixed, regardless of the tensor parallel size. It involves each rank in a tp group reading the full weight file, and...