server
server copied to clipboard
triton-inference-server
Expose tritonserver args in values.yaml
Current k8s_onprem chart has predefined args in deployment.yaml. We could change it from the yaml file though but would be cleaner to customise the args through values.yaml instead.
@rmccorm4 , Thanks for your reply.
- I have updated the naming based on your suggestion.
- I can work out an example. Can advise where to provide this example?
- I will follow up with the CLA.
Thanks.
@rmccorm4, as requested, I have added the examples for sanity check. Any comments?
The examples show how you can set the customised serverArgs and verify that they are taking effect.
- Execute the command below to generate the helm template to check that the serverArgs is being set correctly.
helm template --set serverArgs="{'--model-repository=s3://192.168.1.110:9000/triton-models', '--repository-poll-secs=8', '--model-control-mode=poll', '--grpc-use-ssl=false'}" triton -n triton .
You should see the generated deployment.yaml. The serverArgs are populated as array args in this example.
- Run the container using the command.
helm install --set serverArgs="{'--model-repository=s3://192.168.1.110:9000/triton-models', '--repository-poll-secs=8', '--model-control-mode=poll', '--grpc-use-ssl=false'}" triton -n triton .
Run kubectl describe pods -n triton. You should see the args in the description.
Once the container is successfully running, run kubectl logs <container id> -n triton.
You should see these outputs that indicates the args set for the triton server.
for e.g. model-repository, model-control-mode
for e.g. repository-poll-secs. You can see from the logs that "Polling model repository" happens every 8 secs.
I have also emailed my signed CLA.
Hi @okyspace, can you either resend this email or clarify the name and company attached to it?
@rmccorm4 I have resend the CLA. Name: Ong Kai Yong Company: N/A. I am not submitting on behalf of a company.