jiangtaozh
jiangtaozh
pip show sglang Name: sglang Version: 0.4.10.post1
yes. i upgraded to # pip show sglang Name: sglang Version: 0.5.1 I will let you know if we still have the nan problem.
Met error: https://github.com/sgl-project/sglang/issues/7033 in sgkernel undefined symbol.
root@15e470876a14:/app/SpecForge# pip show sglang Name: sglang Version: 0.5.1 [rank0]: Traceback (most recent call last): [rank0]: File "/app/SpecForge/scripts/prepare_hidden_states.py", line 387, in [rank0]: main() [rank0]: File "/app/SpecForge/scripts/prepare_hidden_states.py", line 383, in main [rank0]:...
The error only happens when tp size is greater than 2. I tested tp=1 and tp=2 both are good.