Chen Shen

Tianjin University Tianjin I am a postgraduate in Tianjin University and a member of Tanklab. I received my B.S. degree in Dalian University of Technology.

Results 2 comments of


                                            Chen Shen

Slow inference performance for large Llama models compared to naive MP

This work is very meaningful. I followed @sgsdxzy and conducted the following test on 3090. Model setup | opt-6.7b 1gpu | opt-6.7b 2gpu | opt-1.3b1gpu | opt-1.3b2gpu | opt-13b 4gpu...

Slow inference performance for large Llama models compared to naive MP

Thank you for your answer.