How can we do the same, using KubeRay, RayCluster and RayServe?

[WinsonSou]

So i've got a kubernetes cluster, installed with KubeRay Operator and created a RayCluster, but how do i then create a manifest for Rayserve to serve, say a Llama2 or Mistral 7b? appreciate your help and thank you in advance!

[asprenger]

Sorry but I'm not working with Kubernetes. Please ask at the Anyscale #Kubernetes Slack channel and ask the question. There is a link to Slack on there website.

[TrieuLe0801]

So i've got a kubernetes cluster, installed with KubeRay Operator and created a RayCluster, but how do i then create a manifest for Rayserve to serve, say a Llama2 or Mistral 7b? appreciate your help and thank you in advance!

I see he set it up as the same as instructed in the document of Anyscale. You can follow this https://docs.ray.io/en/latest/cluster/getting-started.html to try his project. I am trying to set up his project again with this