shuailong comments

Repositories
Issues
Comments

Results 4 comments of


                                            shuailong

[Bug]: deepseek_v2 236B on 8XA100 wrong output vllm==0.5.4

![a8ddbe6c90a61c3c23d06b21a5036607](https://github.com/user-attachments/assets/0aa628dd-eb4f-4b3a-af0f-f6dedad259cd)

[Bug]: deepseek_v2 236B on 8XA100 wrong output vllm==0.5.4

> @shuailong616 Hi! Can you try swapping the `dtype=torch.half` to `dtype=torch.bfloat16`? thank you so much for your replay , use torch.bfloat16 have a currect result.

[Bug]: deepseek_v2 236B on 8XA100 wrong output vllm==0.5.4

![Uploading iwEeAqNwbmcDAQTRBN0F0QDwBrCTfCCdbk_4pQbKkrYcZP4AB9ITCQwDCAAJomltCgAL0gACCvw.png_720x720q90.jpg…]()

[Bug]: deepseek_v2 236B on 8XA100 wrong output vllm==0.5.4

> @shuailong616 Hi! Can you try swapping the `dtype=torch.half` to `dtype=torch.bfloat16`? Hi! I also want to konw why torch.half result wrong reason . looking fowward to your reply