Winson Sou

vLLM on OpenShift/Kubernetes Manifests

1

Does anyone have a sample manifest on how to deploy vLLM on OpenShift or Kubernetes?

How can we do the same, using KubeRay, RayCluster and RayServe?

2

So i've got a kubernetes cluster, installed with KubeRay Operator and created a RayCluster, but how do i then create a manifest for Rayserve to serve, say a Llama2 or...