weekend comments

Repositories
Issues
Comments

Results 2 comments of


                                            weekend

[Bug]: V1 engine fails with offline batched inference code in V0 engine

we observed a same issue when deploy llama 3.3 ERROR 02-17 01:53:55 core.py:210] EngineCore hit an exception: Traceback (most recent call last): ERROR 02-17 01:53:55 core.py:210] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 203,...

[Bug]: V1 engine fails with offline batched inference code in V0 engine

@Fy- I think the issue is fixed with latest code. https://github.com/vllm-project/vllm/issues/13076 I bake the main branch and no issues