lightseq icon indicating copy to clipboard operation
lightseq copied to clipboard

Published 20 hours ago • verified publisher icon bytedance

为什么转换后的HDF5模型，推理时间反而比Hugging Face慢？

Open DidaDidaDidaD opened this issue 2 years ago • 1 comments

为什么转换后的HDF5模型，推理时间反而比Hugging Face慢？原本0.24妙推理一个句子，转换模型后反而到了0.33

DidaDidaDidaD avatar Apr 15 '22 17:04 DidaDidaDidaD

Maybe your GPU doesn't support tensorcore for fp16, you can try to build LightSeq with fp32 mode: ENABLE_FP32=1 pip3 install -e $PROJECT_DI

neopro12 avatar May 30 '22 07:05 neopro12