lightseq icon indicating copy to clipboard operation
lightseq copied to clipboard

README里面的【Speedup of Transformer Inference】是哪个模型的,这边用里面的VIT用例测试的性能只有2.64倍?

Open Wayne-Bfx opened this issue 2 years ago • 4 comments

导出模型:python3 export/huggingface/hf_vit_export.py 测试性能:python3 test/ls_vit.py 环境: V100服务器,torch1.10, python3.7.13,CUDA11.3 性能测试结果:

=========lightseq========= lightseq generating... lightseq time: 0.005731139099225402s lightseq results (class predictions): [1] =========huggingface========= huggingface generating... huggingface time: 0.015130232088267803s huggingface results (class predictions): [1]

Wayne-Bfx avatar Nov 24 '22 03:11 Wayne-Bfx

"Speedup of Transformer Inference" is not the speedup of vit model

hexisyztem avatar Nov 24 '22 06:11 hexisyztem

您好,可以给下【Speedup of Transformer Inference】测试的模型名称,或者该模型的github地址不

Wayne-Bfx avatar Nov 24 '22 06:11 Wayne-Bfx

here: https://github.com/bytedance/lightseq/blob/master/examples/inference/python/export/huggingface/hf_bart_export.py https://github.com/bytedance/lightseq/blob/master/examples/inference/python/test/ls_bart.py

hexisyztem avatar Nov 24 '22 07:11 hexisyztem

非常感谢

Wayne-Bfx avatar Nov 24 '22 07:11 Wayne-Bfx