lightseq
lightseq copied to clipboard
README里面的【Speedup of Transformer Inference】是哪个模型的,这边用里面的VIT用例测试的性能只有2.64倍?
导出模型:python3 export/huggingface/hf_vit_export.py 测试性能:python3 test/ls_vit.py 环境: V100服务器,torch1.10, python3.7.13,CUDA11.3 性能测试结果:
=========lightseq========= lightseq generating... lightseq time: 0.005731139099225402s lightseq results (class predictions): [1] =========huggingface========= huggingface generating... huggingface time: 0.015130232088267803s huggingface results (class predictions): [1]
"Speedup of Transformer Inference" is not the speedup of vit model
您好,可以给下【Speedup of Transformer Inference】测试的模型名称,或者该模型的github地址不
here: https://github.com/bytedance/lightseq/blob/master/examples/inference/python/export/huggingface/hf_bart_export.py https://github.com/bytedance/lightseq/blob/master/examples/inference/python/test/ls_bart.py
非常感谢