Unpacking list templates outside `cmd`

[abhmul]

For one project, I am running a foreach stage, where each step may have a different list of dependencies. I wanted to use templating to fill in the list of dependencies, but it seems for whatever I tried, the syntax was not supported. Below is a simple example that doesn't include the foreach:

# params.yaml
those:
  - script.py
  - script2.py

I first tried

# dvc.yaml
stages:
  hello_world:
    cmd: python script.py
    deps: 
      - ${those}
    outs:
      - hello.txt

Running dvc repro I get

>>> dvc repro
ERROR: failed to parse 'stages.hello_world.deps' in 'dvc.yaml':
Cannot interpolate data of type 'list'

This wouldn't have made that much sense to me anyways since it reads like I'm expanding a list within an entry of another list.

I also tried

stages:
  hello_world:
    cmd: python script.py
    deps: ${those}
    outs:
      - hello.txt

and got

>>> dvc repro
'./dvc.yaml' validation failed.

expected a list, in stages -> hello_world -> deps, line 3, 
column 5
  2   hello_world:                                          
  3 │   cmd: python script.py                               
  4 │   deps: ${those} 

This one makes more sense since it reads similar to how the expansion in foreach stages works.

Please let me know

1. Is what I'm trying to achieve already supported? In that case the templating documentation should be updated, since this behavior is not described anywhere.
2. If this is not currently supported, can this be supported in the future? The latter syntax would be a natural extension of the syntax for templating in foreach stages.

[daavoo]

Hi @abhmul ! unfortunately, this is not currently supported. The same (or very similar) has been discussed at https://github.com/iterative/dvc/issues/7151. Don't hesitate on commenting there your needs/thoughts so we can better prioritize. I will close this one as a duplicate

[dberenbaum]

Reopening per discussion in https://discord.com/channels/485586884165107732/563406153334128681/1157054500494901428 and from #9148, showing that this is a more easily solved issue potentially than #7151.

[slevang]

This would be a huge improvement, would love to see this get implemented. This is the only impediment to reducing my dvc.yaml lines by about 5x and making everything much clearer and more robust within a common foreach. Only because of some variable dependencies between otherwise similar independent branches of the dag, same as @abhmul.

[lefos99]

Hello, any progress on this one? 👀