dstack icon indicating copy to clipboard operation
dstack copied to clipboard

Allow to use interpolation in environment variables

Open kengz opened this issue 1 year ago • 3 comments

It is common practice to append to PATH like PATH=/opt/conda/envs/mnist/bin:$PATH. However using this in the workflow file env seems ~seems to replace PATH literally without variable interpolation for $PATH.~ the workflow does not even run - no log is shown and workflow simply fails.

  - name: check-conda
    provider: bash
    env:
      - PATH=/opt/conda/envs/mnist:$PATH
    commands:
      - echo 'checking $PATH'
      - echo $PATH
      - echo 'done'
      - conda env list # this results in `conda not found`

kengz avatar Sep 15 '22 13:09 kengz

If I understand right, it can be implemented if we instead of passing environment variables directly to the job, the provider will set environment variables via commands ( export ENV=VALUE).

peterschmidt85 avatar Sep 23 '22 10:09 peterschmidt85

yes we can always do that, but the existence of env itself sets the expectation that env setting should be done there, including pattern like:

    env:
      - PATH=/opt/conda/envs/mnist:$PATH

kengz avatar Sep 23 '22 13:09 kengz

@kengz Yes, I mean, we can support this. The provider will handle it automatically.

peterschmidt85 avatar Sep 23 '22 13:09 peterschmidt85