dstack icon indicating copy to clipboard operation
dstack copied to clipboard

[Feature]: Allow simple selection of which instance to use from `dstack run`

Open spott opened this issue 4 months ago • 4 comments

Problem

A user might want to choose between a small subset of the returned instances that they can run a job on due to things like stability, cost, or compute speed among instances that are otherwise equivalent.

Solution

Allow the user to select from some small number of instances that are shown (top 5 maybe?) by calling it by number when they accept the run of a job.

Allow a mechanism for sorting (so --prefer-cheaper, --prefer-gpu-mem, --prefer-gpu-speed, etc maybe with a global config option as well), and then when they are presented with the list of instances, they can choose a number instead of just "y/n". If that instance is no longer available after the user selects, then either reshow a list or just choose the best one left on the list (this can be configured).

Benefit

Sometimes you get multiple instances that are equivalent in some chosen way (price/gpu memory/ cpu memory etc.) however are more desirable along a second axis: for example you want a >=40Gb gpu and if an H100 is available take that, but if not, an A100 or A6000 is fine. Or you want the cheapest node you can get with >20Gb of vram.

Alternatives

At the moment, we can choose along some simple dimensions (max price, characteristic of gpu, etc) which allows for some narrowing of the pool size, but more choice would be better.

Would you like to help contributing this feature?

Yes

spott avatar Feb 28 '24 18:02 spott