dstack
dstack copied to clipboard
dstackai
dstack is an open-source alternative to Kubernetes, designed to simplify development, training, and deployment of AI across any cloud or on-prem. It supports NVIDIA, AMD, and TPU.
### Problem When launching a VM on GCP that uses an image stored in a private GAR the only documented approach for getting access is to use the registry_credentials option...
### Problem Some volume mounts are not crucial for runs, e.g. if they are only used for caching. However, specifying such mounts in the run configuration limits the backends and...
### Problem Currently, if I want to specify TPU resources, I cannot specify the number of GPUs but I have to specify the number of cores. Example: ```yaml # ......
### Problem Some web apps require additional configuration to work with a path prefix. Currently, it's not possible to write a single `type: service` configuration for such apps that would...
### Problem Currently, the gateway installation and updates are performed using a chain of two packages. First, the `dstack-gateway` package is installed from s3. The `dstack-gateway` package depends on the...
### Problem The in-server proxy never removes [SSH connections](https://github.com/dstackai/dstack/blob/39a1c253c7b3101919b8df5d9fde7f9412f3d79a/src/dstack/_internal/proxy/lib/services/service_connection.py#L90) to service replicas, even after the replicas were stopped. While the SSH processes can stop automatically, the connection objects are stored...
### Problem Some services may need to know the IP address of the end user or the entire proxy chain. This can be achieved by having the reverse proxy set...
### Steps to reproduce 1. Start a run (e.g. a dev environment) 2. Stop the run 3. The shim will log an error indicating that the container exited with 137...
### Problem RPS auto-scaling is only supported when running services with a gateway. It is not possible to run a service without a gateway with RPS auto-scaling enabled. ```shell >...
### Problem When a service is running without a gateway, it can only accept regular HTTP requests. Websocket connections are [dropped](https://github.com/dstackai/dstack/blob/39186899940740b7813ec81c7e6d379e48becaeb/src/dstack/_internal/server/services/proxy/services/service_proxy.py#L28). This limits what applications `dstack` can run without a...