dstack
dstack copied to clipboard
[Bug]: Blocks param for SSH fleet does not work at top yaml level
Steps to reproduce
I was trying to create SSH fleet with Tenstorrent chips with blocks: auto so that we can deploy workloads which use less than all chips on the node.
When I create the fleet with dstack apply -f tt-fleet.yaml with blocks: auto defined at the top yaml level
type: fleet
name: tt-fleet
blocks: auto
ssh_config:
user: kopec
identity_file: ~/.ssh/id_ed25519
hosts:
- <IP>
then dev env creation fails with No matching instance offers available as if the configured node was not available.
However when I move blocks: auto to the node level like so
type: fleet
name: tt-fleet
ssh_config:
user: kopec
identity_file: ~/.ssh/id_ed25519
hosts:
- hostname: <IP>
blocks: auto
it works properly
Actual behaviour
This is what I got when blocks: auto was set at the top yaml level
$ dstack fleet
FLEET INSTANCE BACKEND RESOURCES PRICE STATUS CREATED
tt-fleet 0 ssh (remote) cpu=32 mem=504GB disk=2804GB n300:24GB:4 $0 idle 3 mins ago
$ dstack apply -f pawelk-dev-tt.yaml
The working_dir is not set — using legacy default "/workflow". Future versions will default to the image's working directory.
Project main
User admin
Configuration pawelk-dev-tt.yaml
Type dev-environment
Resources cpu=2.. mem=8GB.. disk=100GB.. n300:1
Spot policy on-demand
Max price -
Retry policy -
Creation policy reuse-or-create
Idle duration 5m
Max duration -
Inactivity duration -
Reservation -
No matching instance offers available. Possible reasons: https://dstack.ai/docs/guides/troubleshooting/#no-offers
Expected behaviour
This is the desired output I got once I moved blocks: auto to the node level
$ dstack apply -f pawelk-dev-tt.yaml
The working_dir is not set — using legacy default "/workflow". Future versions will default to the image's working directory.
Project main
User admin
Configuration pawelk-dev-tt.yaml
Type dev-environment
Resources cpu=2.. mem=8GB.. disk=100GB.. n300:1
Spot policy on-demand
Max price -
Retry policy -
Creation policy reuse-or-create
Idle duration 5m
Max duration -
Inactivity duration -
Reservation -
# BACKEND RESOURCES INSTANCE TYPE PRICE
1 ssh (remote) cpu=8 mem=126GB disk=2804GB n300:24GB:1 instance (1/4) $0 idle
Submit the run pawelk-dev-tt? [y/n]: y
NAME BACKEND GPU PRICE STATUS SUBMITTED
pawelk-dev-tt ssh (remote) n300:24GB:1 $0 running 12:13
pawelk-dev-tt provisioning completed (running)
/tmp/vscode-server- 100%[===================>] 64.74M 38.1MB/s in 1.7s
mkdir: created directory '/root/.vscode-server'
mkdir: created directory '/root/.vscode-server/bin'
mkdir: created directory '/root/.vscode-server/bin/2901c5ac6db8a986a5666c3af51ff804d05af0d4'
Installing extensions...
Installing extension 'ms-python.python'...
Installing extension 'ms-toolsai.jupyter'...
Extension 'ms-toolsai.vscode-jupyter-cell-tags' v0.1.9 was successfully installed.
Extension 'ms-python.vscode-python-envs' v1.2.0 was successfully installed.
Extension 'ms-toolsai.jupyter-renderers' v1.3.0 was successfully installed.
Extension 'ms-python.debugpy' v2025.14.1 was successfully installed.
Extension 'ms-toolsai.jupyter' v2023.8.1002501831 was successfully installed.
Extension 'ms-python.python' v2025.16.0 was successfully installed.
Extension 'ms-toolsai.vscode-jupyter-slideshow' v0.1.6 was successfully installed.
Extension 'ms-toolsai.jupyter-keymap' v1.1.2 was successfully installed.
Extension 'ms-python.vscode-pylance' v2025.9.1 was successfully installed.
pip install ipykernel...
To open in VS Code Desktop, use link below:
vscode://vscode-remote/ssh-remote+pawelk-dev-tt/workflow
To connect via SSH, use: `ssh pawelk-dev-tt`
dstack version
0.19.36
Server logs
Additional information
Solved offline with @peterschmidt85. Suspected cause is wrong field validation at global level