skypilot icon indicating copy to clipboard operation
skypilot copied to clipboard

[SERVE] Add the ability to update a service in place

Open JGSweets opened this issue 1 year ago • 4 comments

Currently when pushing an update to a service, if there's an overlap in requirements for resources the new version does not utilize resources from the old version.

It would be ideal that it reuses resources if the following conditions are met:

  1. All other conditions have not changed
    • code
    • storage
    • etc.
  2. There's an overlap in resources that the previous replica is using and the new replica requires
    • e.g. both systems utilize an instance type of X and the previous replica already has X stood up.
    • Therefore, transfer the previous resource to the new replica no longer requiring a new resource to stand up.
    • launch anything else required by the updated service as needed.

EDIT: Example Cases:

  1. Add a new allowed accelerator and keep the previous accelerators.
  2. Update the min / max replicas.

These shouldn't require tearing down previous resources or spinning up new ones where it is unnecessary as this adds extra cost or may not be feasible due to service limit constraints.

JGSweets avatar Feb 05 '24 18:02 JGSweets