Karan Pathak

icon indicating a website link karanpathak.github.io

icon location Vancouver, British Columbia icon location Optimizing my Life and reducing human efforts using machine learning. Currently working on computer vision problems

Results 1 issues of Karan Pathak

vLLM model serving server hangs when GPU KV cache usage reaches 10%

3 comment

Hello everybody, The server hangs when the GPU KV cache usage reaches 10%. ## Issue in Detail I attempted to serve the Llama2 7B Hugging Face model via vLLM on...