ray_vllm_inference icon indicating copy to clipboard operation
ray_vllm_inference copied to clipboard

Published 20 hours ago • verified publisher icon asprenger

how to use when offline LLM?

Open ipv6ok opened this issue 1 year ago • 0 comments

Your tutorial uses an online model download, but some environments are offline. Is there a deployment guide for offline LLM?

ipv6ok avatar Feb 28 '24 09:02 ipv6ok