vllm
vllm copied to clipboard
add policy
Hi @xxw1995, could you elaborate more on this PR (e.g., what it is for and how much performance gain you got)?
Hi @xxw1995, could you elaborate more on this PR (e.g., what it is for and how much performance gain you got)?
In our business, we achieved approximately a 15% performance gain.