truss
truss copied to clipboard
Generate deployment configurations such as k8s deployment specs for Truss
The journey from the ml model to a serving model can be made even easier by generating deployment artifacts such as k8s deployment specs and helm charts.
A k8s deployment manifest could be a good starting point. It will save users the work of figuring out how to deploy a Truss docker image to k8s. Even if they want to customize it, it will provide them a good starting point.
We could start with something basic:
- Generate a deployment spec
- Generate a k8s service spec to invoke the served model
As a follow-up, it would be good to extend support with a section to support common options such as the number of replicas. As much as possible we should capture succinct specifications in the Truss config to auto-generate the k8s specs. But, of course, users can customize the generated deployment specs the way they want.
A helm chart would be a great way of packaging these k8s specs. This should perhaps be captured in a separate issue and done as a follow-up.
We want to integration-test this. For this, we'll need to update the codespace setup to install minikube, to be able to start a local minikube cluster for these tests.