Download artifacts using reusable workflow doesn't work

[shaypi]

In my reusable workflow, I run other workflows after the first has been successfully completed (which include uploading artifacts) .

The artifacts from the first workflow cannot be downloaded for some reason

Run dawidd6/[email protected]
==> Repository: XXXXXXX
==> Artifact name: HaXM_Agent_Build_Windows
==> Local path: ./
==> Workflow name: HaXM_Agent.yml
==> Workflow conclusion: in_progress
==> (found) Run ID: 10622
==> (found) Run date: [20](https://XXXXXXXXXXXXXXXXX/runs/12571?check_suite_focus=true#step:14:20)23-02-08T[21](XXXXXXXXXXXXXXXXXXXXXXXXX/runs/12571?check_suite_focus=true#step:14:21):49:41Z
==> (not found) Artifact: HaXM_Agent_Build_Windows
==> Found the following artifacts instead:
Error: no artifacts found

[dawidd6]

Link to repo?

[shaypi]

@dawidd6 I wish I could, the repo is in a private enterprise account. But I do found that if I add workflow_conclusion: failure it will take the artifact from the previous job that failed

which means that I need to understand with what flag to use in order of taking an on going workflow (I tried in progress and it didn't work)

Can be one of a workflow conclusion:

#   "failure", "success", "neutral", "cancelled", "skipped", "timed_out", "action_required"
# Or a workflow status:
#   "completed", "in_progress", "queued"

[yaozhang09]

were you able to get it to work @shaypi i'm encountering the same problem

[shaypi]

were you able to get it to work @shaypi i'm encountering the same problem

@yaozhang09 No, I didn't. It look like the issue is not related to @dawidd6 workflow. It seems that Reusable workflow uploads the artifacts at the end of the runtime, instead of during the runtime of the job.

[Imeguras]

Well i had a similar conclusion with my ci/cd pipeline

Here's the setup im using: it all starts with a main,

name: main_wf
on:
  push: 
    branches: [ master, workflow-stuff ]
jobs:
  test-qt-desktop:
    uses: 
      ./.github/workflows/testQTdesktop_wf.yml
    secrets: inherit

which then flows to testQTdesktop

name: testQTdesktop_wf
on:
  workflow_call:
    inputs:
      suite: 
        required: false
        type: string
jobs: 
  build-qt-desktop:
    uses: ./.github/workflows/buildQmake_wf.yml
    with:
      qmakeflags: "CONFIG+=test"
      binaryfilename: VolanteAlphaQT_testes
  test-qt-desktop:
    needs: [build-qt-desktop]
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Fetch artifacts from build-qt-desktop
        uses: dawidd6/action-download-artifact@v2
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          workflow: ${{ github.event.workflow_run.workflow_id }}
          name: VolanteAlphaQT_testes
      - name: Run tests
        run: cd $GITHUB_WORKSPACE && ./VolanteAlphaQT -platform offscreen

in the middle a workflow is hit and produces an artifact

name: buildQmake_wf.yml
on: 
  workflow_call:
    inputs:
      qmakeflags:
        required: false
        type: string
      binaryfilename:
        required: true
        type: string      
jobs: 
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout
      uses: actions/checkout@v3
    - uses: awalsh128/cache-apt-pkgs-action@latest
      with:
        packages: qt6-base-dev qt5-qmake qtbase5-dev libqt6serialport6 libqt6serialport6-dev libqt5serialport5 libqt5serialport5-dev nlohmann-json3-dev socat gengetopt
        version: 1.0
    - name: Build with qmake
      run: /usr/bin/qmake VolanteAlphaQT.pro ${{ inputs.qmakeflags }} && make
    - name: Upload artifact
      uses: actions/upload-artifact@v2
      with:
        name: ${{ inputs.binaryfilename }}
        path: ${{ github.workspace }}/bin/${{ inputs.binaryfilename }}
        if-no-files-found: error

The build step runs as it should producing an artifact with the name VolanteAlphaQT_testes the thing is the download part results in:

Run dawidd6/action-download-artifact@v2
  with:
    github_token: ***
    path: $GITHUB_WORKSPACE
    name: VolanteAlphaQT_testes
    workflow_conclusion: success
    repo: FSIPLEIRIA/volante_alpha_qt
    check_artifacts: false
    search_artifacts: false
    skip_unpack: false
    if_no_artifact_found: fail
==> Repository: FSIPLEIRIA/volante_alpha_qt
==> Artifact name: VolanteAlphaQT_testes
==> Local path: $GITHUB_WORKSPACE
==> Workflow name: 49617406
==> Workflow conclusion: success
==> (found) Run ID: 4285452593
==> (found) Run date: 2023-02-27T18:09:02Z
==> (not found) Artifact: VolanteAlphaQT_testes
==> Found the following artifacts instead:
Error: no artifacts found

Am i doing something wrong? Is this behaviour supported by this repo?

[Imeguras]

@dawidd6 moved the issue so you can erase the old one i guess

[elalish]

I just started running into this as well, oddly three hours ago it was working, but now (n=2) it is not. Here is where the failure message is, and here is the successful upload (under the "Upload WASM files" step).

[Imeguras]

No solutions yet? i guess ill just make a spaghetti script

[elalish]

The problem hasn't come up again for me, so I'm guessing it was some kind of Github outage.

[shaypi]

@elalish what you showed up is not the same case scenario which I'm talking about

[Imeguras]

@lukasz-mitka solved my issue Thank you!

[lukasz-mitka]

@Imeguras how did I do that?

[Imeguras]

@Imeguras how did I do that?

Well sorry i thought you did the changes although you still reviewed it

[jbrsb]

Hello @shaypi, did you find a workaround? I'm facing the same issue despite it was working previously. Workflow A calling workflow B building/uploading artifacts then workflow C called by A try to download theses artefacts. Was working with workflow_conclusion in_progress. I cannot use classic download/artefact because I need to download specific artifacts with dynamical name.

[ishanjain28]

Github's documentation on working with artifacts says,

Artifacts are uploaded during a workflow run,

and

During a workflow run, you can use the download-artifact action to download artifacts that were previously uploaded in the same workflow run.

But after testing, It looks like it only uploads artifacts(or those artifacts are only returned by the API) after the execution has finished. This feels like a bug to me but I am not sure, I'll create a ticket with github and ask them for explanation.

https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts

[shaypi]

I'm facing the same issue despite it was working

Hi @jbrsb, I have discovered two effective workarounds. The first approach involves uploading the files to a designated storage bucket and subsequently retrieving them in the next job. Alternatively, you can utilize GitHub releases by uploading the assets to a specific release and then retrieving them in the subsequent job.

[dawidd6]

If you are using reusable workflows you need to set workflow input to the calling workflow filename instead of the one where artifact is actually uploaded. Also you need to set workflow_conclusion input to either an empty string or "in_progress".

[Jimbly]

Thanks for the info here, it helped lead me in the right direction. Just to add to this for anyone who gets here facing similar problems, I have a workflow that is either ran as part of a reusuable workflow or is ran directly (I suspect similar issues if your workflow is reused by multiple reusable workflows). workflow ends up being one of a few different values, workflow_conclusion ends up being either success or in_progress depending on whether the last successful run was part of the current reusable workflow or an independent run. For my example, I'm uploading an artifact named last_build.json during a "build" workflow, and then downloading it later during a "deploy" workflow (sometimes build once deploy multiple times, sometimes want to just do a build for a later deployment, etc).

In my "build" workflow, after a successful build, I save the artifact:

    - name: Upload image build artifact
      uses: actions/upload-artifact@v4
      with:
        name: last-build-artifact
        path: |-
          last_build.json

Then the "deploy" workflow, which is hit either as a workflow_call in a run that may or may not have had a "build" workflow, and also as a manual workflow_trigger:

name: deploy.yml

on:
  workflow_call:
    inputs:
      example:
        default: 'example'
        type: string
  workflow_dispatch:
    inputs:
      example:
        default: 'example'
        type: string
jobs:
  doit:
    steps:
    - name: Download last_build.json from uploaded artifacts
      uses: dawidd6/[email protected]
      with:
        workflow_search: true # either deploy.yml or some calling workflow
        workflow_conclusion: '' # either 'success' (if a previous run) or 'in_progress' (if part of the current workflow)
        name: last-build-artifact
        search_artifacts: true

    # Make sure we have successfully downloaded last_build.json artifact
    - name: Check if last_build.json exists
      run: |-
        ls -a
        if [ ! -f ./last_build.json ]
        then
          echo "Could not locate last_build.json. Make sure to run build.yml workflow before this! Exiting job."
          exit 1
        else
          echo "Successfully located last_build.json"
          cat last_build.json
        fi