ipex-llm icon indicating copy to clipboard operation
ipex-llm copied to clipboard

Published 20 hours ago • verified publisher icon intel-analytics

6K input OOM on ARC with VLLM-serving

Open jessie-zhao opened this issue 7 months ago • 1 comments

Faced OOM on Arc with 6k input/512 out with VLLM serving, Mode: ChatGLM3-bB, Qwen1.5-32B on 4 ARC

jessie-zhao avatar Jul 11 '24 07:07 jessie-zhao