vllm icon indicating copy to clipboard operation
vllm copied to clipboard

Published 20 hours ago • verified publisher icon vllm-project

[Hardware][Intel-Gaudi] Regional compilation support

Open Kacper-Pietkun opened this issue 1 week ago • 1 comments

Add support for regional compilation for HPU. It is turned on by default, but can be turned off with VLLM_REGIONAL_COMPILATION env variable. It works only for torch.compile execution mode. It significantly speeds up warmup time and slightly increases throughput.

Signed-off-by: Kacper Pietkun [email protected]

Kacper-Pietkun avatar Feb 13 '25 09:02 Kacper-Pietkun