Add GPU Placement and Sweeps Queuing

[tchaton]

🚀 Feature

Motivation

After chatting with possible early adopters like Georgehwp, the Sweep is GPU placement and Queuing from his side as he would like to use it on his own machine or machnes.

GPU Orchestration: As a user with 8 GPUs, I would like to App to place the trials on the available GPUS automatically.

Queuing Mechanism: Assuming all GPUS are used, I would like the Sweeps to wait for GPU to be available before starting.

Pitch

Alternatives

Additional context

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tshu-w]

This can be easily achieved using the ray tune, as ray contains control over various resources and ray tune can replace optuna.