JGSweets

[Core][AWS] Allow specification of Security Groups for resources.

3

Similar to https://github.com/skypilot-org/skypilot/pull/3488 but for SGs This PR: * Allows specification of SGs for controllers vs workers. * **Currently removes the port requirement to be able to allow SG specification...

[Core][AWS] Allow the config to set IAM roles for resources being launched by AWS

4

Skypilot forces all of its created resources to utilize `skypilot-[v1]` IAM role. The features requested: 1. Allow specification of the IAM role 2. Allow specification of which resources get which...

[Core][Serve] Cannot specify security groups for the serve nodes.

2

Recently added in https://github.com/skypilot-org/skypilot/issues/1354, we can specify the default `security_group_name` for aws resources. However, when launching serve, there are two conflicts which occur between the serve config requirements and the...

[BUG] workers do not launch on g5.12xlarges for the latest image 0.5.0.

6

I'm stuck in a repeat deployment loop when utilizing the image `anyscale/ray-llm:latest` on a g5.12xlarge instance. It seems the worker never connects back which leads me to believe an error...

Need a TypeScript client with the same API as the python client

1

**Is your feature request related to a problem? Please describe.** Currently, there's only a python client. **Describe the outcome you'd like:** Similar to the python client here: `https://github.com/capitalone/federated-model-aggregation/tree/main/clients` We need...

Checksum validation with hf_hub_download on model files.

5

**Is your feature request related to a problem? Please describe.** After reviewing: #1738 and #2223 it looks like file checksums are only computed on the cache dir in specific conditions....

[Serve] Allow blue-green deployments of the controller / full cluster.

1

Currently, when running `update ...`, or even `update --mode blue_green ...`, the controller doesn't update. Currently, one has to start a new service and swap over. This also changes its...

Currently, replicas scale based on QPS. It would also be beneficial to be able to scale based on response time. This can help ensure a maximum response time when a...

Currently (v0.6.0), the load balancer logs contain the key `headers` from requests. https://github.com/skypilot-org/skypilot/blob/v0.6.0/sky/serve/load_balancing_policies.py#L15 These values may contain sensitive data that are not desired to be stored in logs. Ideally, we...

[BUG][AWS] Serving a new controller can result in terminating previously served models.

2

When these conditions are true, a new controller may terminate existing resources being served by another controller. * User id hash matches last 4 characters, but has a difference otherwise....