lightseq.inference has no attribute 'bert'

[boxiaowave]

I tried to install the fp32 version by building from source. After finishing the installation of cmake, hdf5 and so on, I ran the conmmand 'PATH=/usr/local/hdf5/:$PATH ENABLE_FP32=1 ENABLE_DEBUG=1 pip3 install -e ./' in the root dir of lightseq to install lightseq, however I met the error 'lightseq.inference has no attribute 'bert'' when I tried to run the inference sample, are there any steps I missed?

[Taka152]

You can try pip3 install -v -e ./ to check if the build is successful.

[boxiaowave]

You can try pip3 install -v -e ./ to check if the build is successful.

thanks, it's my mistake, the version of cmake is incorrect.

[boxiaowave]

You can try pip3 install -v -e ./ to check if the build is successful.

I found anther problem, is the model file exported by export_lighseq.py in a fp16 type? The output logits of huggingface model and lightseq model are different on 0.001 scale when I use fp32 inference mode.

[Taka152]

No, the weights are saved in fp32. 1e-4 precision loss in fp32 mode is acceptable. You try a higher level metric to measure the difference.

---

发件人: SamuelSiu @.> 发送时间: Tuesday, February 15, 2022 3:48:06 PM 收件人: bytedance/lightseq @.> 抄送: Ying Xiong @.>; Comment @.> 主题: Re: [bytedance/lightseq] lightseq.inference has no attribute 'bert' (Issue #266)

You can try pip3 install -v -e ./ to check if the build is successful.

I found anther problem, is the model file exported by export_lighseq.py in a fp16 type? The output logits of huggingface model and lightseq model are different on 0.001 scale when I use fp32 inference mode.

― Reply to this email directly, view it on GitHubhttps://github.com/bytedance/lightseq/issues/266#issuecomment-1039955164, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AELIZAJTA57JZTMRBUZO3L3U3IALNANCNFSM5OKSBOEQ. You are receiving this because you commented.Message ID: @.***>