Inject environment variables into the sidecar for configuring AI Runtime
🚀 Feature Description and Motivation
Multiple types of information are required during the startup process of AI Runtime: -Engine side: inference engine type, access address for engine metrics -Model side: Remote address of the model to be loaded, download authentication information
The above information needs to be injected into the AI Runtime container through environment variables.
Use Case
No response
Proposed Solution
It is possible to pass the required information in the annotation and inject it into the environment variables of the side car, similar to the implementation of Istio metrics merge
We may build a top level orchestration at this moment so we can orchestrate the deployment by ourselves. This is on hold
Remove it from v0.2.0. Pending on model orchestration. Now, the easier way is to configure the S3 env in runtime container.. Do not let engine to handle it.