runner icon indicating copy to clipboard operation
runner copied to clipboard

Workflow level `env` does not work properly in all fields.

Open timharris777 opened this issue 5 years ago • 37 comments

Describe the bug When using workflow level environment variables they do not work when I use them to substitute for jobs.build.name or jobs.build.runs-on. See workflow file below for example:

name: Testing workflow leve env vars
on:
  push:
    branches:
      - develop

env:
  aws_account: cfaittoolsnp
  environment: dev

jobs:
  build:
    name: Deploy|${{ env.aws_account }}
    runs-on: ${{ env.aws_account }}
    steps:
    - name: Checkout
      uses: actions/checkout@v2
    - name: Init
      run: |
        echo ${{ env.aws_account }}

To Reproduce Steps to reproduce the behavior: 1.- Run the workflow file above to receive the following error: The workflow is not valid. .github/workflows/dev.yml (Line: 15, Col: 11): Unrecognized named-value: 'env'. Located at position 1 within expression: env.aws_account,.github/workflows/dev.yml (Line: 16, Col: 14): Unrecognized named-value: 'env'. Located at position 1 within expression: env.aws_account

Expected behavior The expected behavior is that using the workflow file above the build.name would be Deploy|cfaittoolsnp and the build.runs-on would be cfaittoolsnp.

Runner Version and Platform

Running latest version of linux self-hosted runner.

OS of the machine running the runner? linux (ubuntu 18.04)

timharris777 avatar May 15 '20 13:05 timharris777

I'm just wondering if there is any update on this? :)

dgflexera avatar Jan 21 '21 06:01 dgflexera

Unfortunately, no-- env can still not be used. However, I have found that I can use either of the following:

  • runs-on: ${{ needs.init.outputs.tag }} -> You would need to export the tag for the runner as an output in a previous step.
  • runs-on: ${{ github.event.inputs.tag }} -> You would need to provide an input for the runner tag via worfklow_dispatch.

timharris777 avatar Jan 26 '21 17:01 timharris777

I'm new to GitHub, hence GitHub workflows. But this is the exactly the first issue I have with workflow. I'd image this is causing troubles for many of us. Does anyone know the best way to attract attentions from GitHub on this?

leonardchu avatar Feb 16 '21 23:02 leonardchu

@leonardchu, give the first post a thumbs up. Github says that they monitor that. Other than that I'm not sure.

timharris777 avatar Feb 17 '21 14:02 timharris777

Same issue with env in job strategy:

name: Python package

on: [push, pull_request]

env:
  target_python_version: 3.8

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      max-parallel: 5
      matrix:
        python-version: [3.7, "${{env.target_python_version}}"]

    steps:
    - uses: actions/checkout@v1
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install tox tox-gh-actions
    - name: Test with tox
      run: tox
.github/workflows/main.yml#L14The workflow is not valid. .github/workflows/main.yml (Line: 14, Col: 31): Unrecognized named-value: 'env'. Located at position 1 within expression: env.target_python_version

pdecat avatar Mar 20 '21 08:03 pdecat

Hello! I'm having the same issue in the following context:

step:
  - uses: ${{ env.org }}/repo/my_action@master

adrienbernede avatar Apr 09 '21 18:04 adrienbernede

Same problem for me with if: ${{ github.ref != env.version }}

roycdiscovery avatar May 11 '21 13:05 roycdiscovery

It looks like instead of env a strategy.matrix variable with one value can be used in some cases.

rvernica avatar Jul 13 '21 19:07 rvernica

Any workarounds on this? I can't use env on workflow level as well :(

fabasoad avatar Oct 10 '21 11:10 fabasoad

Any workarounds on this? I can't use env on workflow level as well :(

mine - for https://github.com/nektos/act :

diff --color -u -r .github/workflows/use-github-actions.yml .act-github/workflows/use-github-actions.yml
--- .github/workflows/use-github-actions.yml	2021-10-02 15:45:43.502705340 +0300
+++ .act-github/workflows/use-github-actions.yml	2021-10-02 15:45:43.516705231 +0300
@@ -2,6 +2,7 @@
     test-fc-solve:
         env:
             WHAT: ${{ matrix.env.WHAT }}
+        if: ${{ ! contains(env.ACT_SKIP, matrix.env.WHAT) }}
         runs-on: ubuntu-latest
         steps:
         -   uses: actions/checkout@v2
Only in .github/workflows: windows-x64.yml
Only in .github/workflows: windows-x86.yml

shlomif avatar Oct 10 '21 12:10 shlomif

This is quite the dealbreaker for a dynamic workflow :-S I hope this can be resolved asap.

@shlomif looks promising, but I can't quite figure out what you're doing there. Could you provide a bit more context?

oli414 avatar Oct 24 '21 21:10 oli414

@oli414 : I'm using an envvar to skip workflows, but only in the "act" configuration (where there isn't a problem) E.g:

act --workflows .act-github/workflows --rm --env-file <(echo "ACT_SKIP=,buildproc,")

shlomif avatar Oct 25 '21 04:10 shlomif

Maybe env is not the right mechanism for some of these use cases. What I'd like to see at least is for the Actions YAML parser to support aliases and anchors in order to create reusable elements in the workflow file.

restjohn avatar Nov 10 '21 14:11 restjohn

Am also running into this issue trying to use env variables in matrix:

env:
  MIN_PYTHON_VER: 3.6.7
  LATEST_PYTHON_VER: 3.10.0

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        python-version:
          - ${{ env.MIN_PYTHON_VER }}
          - ${{ env.LATEST_PYTHON_VER }}

The fact that one has to hard-code values for things like language and os versions in a separate place, rather than being able to DRY and define them once and reuse them, is really disappointing.

Also disappointing that this is the 7th most liked issue yet has not received a single response from Github in over a year... image

studioph avatar Nov 15 '21 00:11 studioph

@paulhutchings, in pyTooling/Actions we use a Params job to generate an object of "global parameters" which we can then use in any job. See:

    needs:
      - Params
    with:
      artifact: ${{ fromJson(needs.Params.outputs.params).artifacts.coverage }}

fromJson(needs.Params.outputs.params) is certainly ugly, but it is usable in places where env.params fails. Moreover, since we generate the params dynamically, we can declare them imperatively (say Python) and not just declaratively; we can also filter them and/or produce warnings.

That is not a solution to this issue, but at least a usable workaround.

Find a more complex variant of this approach in https://github.com/actions/runner/issues/1182#issuecomment-982696462, where a YAML file with anchors is used for generating the job matrix dynamically.

umarcor avatar Nov 30 '21 14:11 umarcor

Maybe env is not the right mechanism for some of these use cases. What I'd like to see at least is for the Actions YAML parser to support aliases and anchors in order to create reusable elements in the workflow file.

@restjohn, did you see #1182 and actions/starter-workflows#245 ?

umarcor avatar Nov 30 '21 14:11 umarcor

Very upsetting, I wasted two hours on that. And it's not just custom env variables, I couldn't use GITHUB_REF_NAME into zip-release action...

vitalykarasik avatar Jan 03 '22 10:01 vitalykarasik

This is also an issue with reusable workflow. For example:

env:
  DOTNET_VERSION: 5.0.x    

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  call-workflow-passing-data:
      uses: ....reusable.yml@main
      with:
          dotnetversion: ${{ env.DOTNET_VERSION }}`

Does not work.

tjcorr avatar Jan 25 '22 15:01 tjcorr

@tjcorr +1 for not working with reusable workflow. I'm having the same problem.

rafaelfcsouza avatar Jan 26 '22 16:01 rafaelfcsouza

Any news on this? Really annoying on reusable workflows :(

AndreAlves3 avatar Feb 25 '22 09:02 AndreAlves3

Also hitting this trying to use env vars to make a workflow job name dynamic.

Can't do this:

jobs:
  build:
    name: Deploy to ${{ env.GITHUB_REF_NAME }}

But can do this:

jobs:
  build:
    name: Deploy to ${{ github.ref_name }}`

mrfelton avatar Feb 27 '22 13:02 mrfelton

The concept to realize, I think, is that the intent of environment variables is for them to be available in the context that a job step is executing, not in the context of DEFINING a job, which is basically the context of the GitHub Actions platform itself. If you need dynamically computed values for job definition elements, you need to compute them in a step as outputs, as some have commented above, then reference them from subsequent steps. Now, there might be potential for a less cumbersome solution than having to use outputs and needs references, such as user-defined variables and functions that can augment the job definition context, but env is certainly not the correct place for that.

restjohn avatar Feb 27 '22 21:02 restjohn

I took what @umarcor suggested, and I think I've got a nice workaround which allows env vars to be shared in the way we're all looking for:

env:
   FOO: foo
   BAR: bar
jobs:
    params:
        name: Make environment variables available to the rest of the workflow
        runs-on: ubuntu-latest
        outputs:
            params: ${{ steps.env-vars.outputs.params }}
        steps:
            -   name: install jq
                run: pip install jq
            -   id: env-vars
                name: Output environment variables
                run: echo "::set-output name=params::$(echo $(jq -n 'env'))"

    setup:
        needs: [ params ]
        name: Setup
        runs-on: ubuntu-latest
        steps:

            -   name: Use Foo
                run: echo ${{ fromJson(needs.params.outputs.params).FOO }}
            
            -   name: Use Bar
                run: echo ${{ fromJson(needs.params.outputs.params).BAR }}

Essentially we use jq to parse out the env vars from the params step, which inherit the initial env values, and saves this as an output you can use in other jobs. FYI it includes all the other env var junk, I'm sure you can filter it out if needs be.

This solution seems to work for our usecase.

matthew-gill avatar Mar 01 '22 12:03 matthew-gill

My use case is to call a reusable workflow, which requires some secrets as input and you can only use secrets in the job level, so I had to move the uses to the job level, but my reusable workflow itself also requires some inputs, which is definited in my workflow env.... so now I can not pass those envs to my reusable workflow because of this limitation...

neekey avatar Mar 12 '22 14:03 neekey

@restjohn wrote:

The concept to realize, I think, is that the intent of environment variables is for them to be available in the context that a job step is executing, not in the context of DEFINING a job, which is basically the context of the GitHub Actions platform itself. [...] Now, there might be potential for a less cumbersome solution than having to use outputs and needs references, such as user-defined variables and functions that can augment the job definition context, but env is certainly not the correct place for that.

Thanks for your input :)

It may be true that that is how the implementation is implemented and intended currently (or not true, who knows?)

But: The sole purpose of an implementation of software is to serve the users needs, it has no purpose of its own.

Therefore, if it cannot serve the needs of the users, it has to change.

It should not be the other way around, the users should not be asked to change their needs to follow the needs of the implementation :)

xor-freenet avatar Mar 14 '22 22:03 xor-freenet

I don't get it, is the documentation wrong or it does not explains much that we can't use env in reusables workflow actions? Please update the github actions documentation regarding envs limitations.

Devofure avatar Apr 12 '22 18:04 Devofure

I am having the same issue with using env in jobs.<job_id>.container, which the Context Availability documentation states should be available.

adamjseitz avatar Jun 03 '22 13:06 adamjseitz

This is also causing problems for my team's workflows. Do you know when it will be prioritized?

paige-wdc avatar Jun 28 '22 18:06 paige-wdc

We are also running in this issue. What is the latest whereabout on this?

chrisurf avatar Jul 05 '22 12:07 chrisurf

We are also running in this issue. What is the latest whereabout on this?

Still the same. It doesn't work in different places where you would expect it to, and it's still not well documented exactly where it does and doesn't work.

timharris777 avatar Jul 05 '22 13:07 timharris777