truss icon indicating copy to clipboard operation
truss copied to clipboard

Generate deployment configurations such as k8s deployment specs for Truss

Open pankajroark opened this issue 1 year ago • 0 comments

The journey from the ml model to a serving model can be made even easier by generating deployment artifacts such as k8s deployment specs and helm charts.

A k8s deployment manifest could be a good starting point. It will save users the work of figuring out how to deploy a Truss docker image to k8s. Even if they want to customize it, it will provide them a good starting point.

We could start with something basic:

  1. Generate a deployment spec
  2. Generate a k8s service spec to invoke the served model

As a follow-up, it would be good to extend support with a section to support common options such as the number of replicas. As much as possible we should capture succinct specifications in the Truss config to auto-generate the k8s specs. But, of course, users can customize the generated deployment specs the way they want.

A helm chart would be a great way of packaging these k8s specs. This should perhaps be captured in a separate issue and done as a follow-up.

We want to integration-test this. For this, we'll need to update the codespace setup to install minikube, to be able to start a local minikube cluster for these tests.

pankajroark avatar Mar 12 '23 19:03 pankajroark