Simon Mo

Hi @sgolebiewski-intel, thank you for the PR. currently it is hard to distinguish the changes that are changing the grammar/style vs breaking up sentences into multiple line in markdown. Is...

Please see the failed documentation build at https://buildkite.com/vllm/ci-aws/builds/7975#0191b133-34dd-4462-bcd4-6d19d79ebbc9

@abmfy Thank you can you fix this PR? This is part of the release blocker now.

cc @youkaichao if you have any suggestions but given this is a harder to reproduce nccl segfault, i recommend setting up fault tolerance for the service for now.

@youkaichao @ShangmingCai, this seems to break the tests https://buildkite.com/vllm/ci/builds/13743#01951f87-e956-47dc-8c32-3faee9c97af1/6-12975 with error ``` [2025-02-19T20:40:07Z] /usr/local/lib/python3.12/dist-packages/vllm/distributed/parallel_state.py:744: AssertionError --   | [2025-02-19T20:40:07Z] _________________ test_init_device[typical_acceptance_sampler] _________________   | [2025-02-19T20:40:07Z]   | [2025-02-19T20:40:07Z] acceptance_sampler_method = 'typical_acceptance_sampler'   | [2025-02-19T20:40:07Z]  ...

Yes the flag sounds natural to me. A more complicate change here will be while outlines fsm is compiling, use lmformatenforcer. But just using flags should be fine as a...

@noamgat engine args is the right place to put it. Do we need it in model config still?

@br3no Yes. Thank you for your suggestion and pushing this through.

Hi @jiqing-feng, Thank you for sending this PR and demonstrating its performance. I have the following feedback: * We are not comfortable with the changes to the build process, vllm/_custom_ops.py,...

Also please try the latest patch v0.5.0.post1 which might fix one of the root cause