xpk icon indicating copy to clipboard operation
xpk copied to clipboard

Make XPK Handle multiple slice sizes

Open rwitten opened this issue 1 year ago • 3 comments

N queues, 1 per slice size, 1 cluster.

(This is complicated!)

rwitten avatar Nov 17 '23 00:11 rwitten

From my understanding I believe this would be multiple resource flavors with their respective chip sizes still in 1 ClusterQueue. https://kubernetes.io/blog/2022/10/04/introducing-kueue/#example-use-case

Obliviour avatar Nov 30 '23 23:11 Obliviour

maybe looks something like this:

xpk cluster create --cluster=my-cluster --tpu-types v5p-128,5 v5p-256,5   # type, num_slices
  • make sure that cluster create when rerunning the command is aware of the heterogenousness of the cluster

Obliviour avatar Dec 07 '23 00:12 Obliviour

One aspect to this is that the overall goal here is to allow the gke cluster to figure out the needed slice types based on a set of user provided options: chip-budget and what incoming requests are.

Probably can also set some minimum / starting points.

Obliviour avatar Jan 16 '24 18:01 Obliviour