aibrix icon indicating copy to clipboard operation
aibrix copied to clipboard

Inject environment variables into the sidecar for configuring AI Runtime

Open brosoul opened this issue 1 year ago • 2 comments

🚀 Feature Description and Motivation

Multiple types of information are required during the startup process of AI Runtime: -Engine side: inference engine type, access address for engine metrics -Model side: Remote address of the model to be loaded, download authentication information

The above information needs to be injected into the AI Runtime container through environment variables.

Use Case

No response

Proposed Solution

It is possible to pass the required information in the annotation and inject it into the environment variables of the side car, similar to the implementation of Istio metrics merge

brosoul avatar Sep 12 '24 06:09 brosoul

We may build a top level orchestration at this moment so we can orchestrate the deployment by ourselves. This is on hold

Jeffwan avatar Sep 25 '24 20:09 Jeffwan

Remove it from v0.2.0. Pending on model orchestration. Now, the easier way is to configure the S3 env in runtime container.. Do not let engine to handle it.

Jeffwan avatar Dec 10 '24 18:12 Jeffwan