dstack
dstack copied to clipboard
dstackai
dstack is an open-source alternative to Kubernetes, designed to simplify development, training, and deployment of AI across any cloud or on-prem. It supports NVIDIA, AMD, and TPU.
See if we still need to build manually vllm, flash-attention, and xformers wheels). Plus update to newer ROCm version.
### Steps to reproduce 1. Create an Azure gateway. ```shell > cat gateways/azure.dstack.yml type: gateway name: azure backend: azure region: westeurope domain: azure.example.com > dstack apply -y gateways/azure.dstack.yml ``` 1....
### Steps to reproduce We have private images available on our fleet ssh hosts. If we want to use those (without going thorough a registry) we cannot start our services....
### Problem Oracle Cloud provides on-demand bare metal 8xH100 instances ( `BM.GPU.H100.8`). Currently, these instances cannot be provisioned with dstack. ### Solution Support the `BM.GPU.H100.8` shape. ### Known challenges 1....
**Problem:** We have at least two problems with SSH fleet instances: 1. [Inderect problem] `dstack` server connects to SSH fleet instances every 4 seconds to check shim health. This alone...