aibrix icon indicating copy to clipboard operation
aibrix copied to clipboard

Implement model architect aware scheduling policies

Open Jeffwan opened this issue 1 year ago • 1 comments

🚀 Feature Description and Motivation

Currently, runtime picks up the work to download the model weights. If we have another replica wants to be deployed, one option is to be scheduled to the same node already has weights. In this case, it's would be great that we can contribute some scheduler plugin to be aware of the artifacts.

Use Case

No response

Proposed Solution

No response

Jeffwan avatar Sep 19 '24 23:09 Jeffwan

It should be done in cold start manager or some other reusable component.

Jeffwan avatar Nov 19 '24 18:11 Jeffwan