Feature request: wildcard support (or download-all)

[StoneCypher]

One of the really useful things about uploading artifacts is caching logs, coverage reports, et cetera. Those often have timestamps or UUIDs as names.

You can hack around it with a compression to a known name step, but, uploading and downloading artifacts by wildcard (or even just download-all) would be a huge benefit for post-processing, eg coverage merging

[DrSensor]

Just want to add a my use case for this :slightly_smiling_face:

build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        target: [tmLanguage, sublime-syntax]
    steps:
    # ...several steps for producing an artifact
    - uses: actions/upload-artifact@master
      with: { name: '${{ matrix.target }}', path: dist }

publish:
    runs-on: ubuntu-latest
    needs: build
    steps:
    - uses: actions/download-artifact@master
      with: { name: 'tmLanguage|sublime-syntax', path: dist }
    # ...several steps for publishing the artifacts

alternatively, maybe make the name field can accept array of string

    - uses: actions/download-artifact@master
      with: { name: [tmLanguage, sublime-syntax], path: dist }

[marble-sh]

alternatively it could mimic the run syntax for jobs.<id>.steps.shell

      - name: Download build artifacts
        uses: actions/download-artifact@v1
        path: dist
        with: |
          foo1.zip
          foo2.zip
          foo3.zip
          foo4.zip
          foo5.zip

[datapythonista]

Would also find this very useful.

In my case I want to upload the logs of all failing jobs as artifacts. And in a last job that needs the rest to finish, I want to download all the job_output_* artifacts, and write the errors to a comment in the PR.

Having to upload/download all the outputs, and specify them manually in a step for each, makes the code much more complex and verbose than needed, and it's a big waste of resources.

[Frontear]

Agreed. My generated code uses naming conventions not accessible to the GitHub action configuration (name-version, of which the action config can never know the version). Allowing for wildcard or even regex naming checks would make this significantly easier.

[dawnbass68]

Gosh Thanks for your help I do appreciate it

On Mon, Mar 2, 2020, 11:48 AM Ali Rizvi [email protected] wrote:

Agreed. My generated code uses versioning not accessible to the GitHub action configuration. Allowing for wildcard or even regex naming checks would make this significantly easier.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/actions/download-artifact/issues/6?email_source=notifications&email_token=AORJ4OAKPATPSRMBWEBTH7TRFQERFA5CNFSM4IUWXTG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENQWCUI#issuecomment-593584465, or unsubscribe https://github.com/notifications/unsubscribe-auth/AORJ4OBBU4ZRRSNXFX2MY6LRFQERFANCNFSM4IUWXTGQ .

[konradpabjan]

v2-preview is out and there is a new option that allows you to download all artifacts at once.

See the pinned issue for more information: https://github.com/actions/download-artifact/issues/23

[ElonH]

I hack the code and implement wildcard support (via RegExp).

https://github.com/ElonH/download-artifact-regexp

But sorry for my ugly code, it is not compatible with upstream.

[konradpabjan]

download-artifact@v2 has been released so you can now download all artifacts at once.

It's not ideal (some users want a subset of artifacts instead of all of them), so I'll keep this issue open until we add wildcard support or something similar.

[fbidu]

Hello all! Hoping you're all safe in the middle of this - seemingly never ending - pandemic.

I'm just passing by to bump this issue. I'm migrating a devops pipeline from Azure Pipeline to GitHub Actions and this is a crucial feature that's missing.

My use case is not new in itself - basically a job with a matrix strategy generates lots of files named <random>.coverage, then another job should download all files named *.coverage into a folder and process them. This is something Azure supplies as this patterns argument

[TWiStErRob]

Hi there, newfound Actions fan here, just want to add my use case after upvoting all that I found above :)

• multiple jobs in workflow, most of them publish an artifact (some of them are big or have many files)
• jobs upload JUnit XML results
• a job at the end that downloads said JUnit XMLs and generates a report

This last job unnecessarily downloads and extracts all artifacts, only to then ignore them with a pattern.

See it as YAML... [click]

jobs:
  other-job1:
    steps:
      ...
      - uses: actions/upload-artifact@v2

  other-job2:
    steps:
      ...
      - uses: actions/upload-artifact@v2

  check:
    env:
      JOB_NAME: Run ${{ matrix.param1 }} on ${{ matrix.param2 }}
    steps:
      - ...
      - uses: actions/upload-artifact@v2
        if: always()
        with:
          name: ${{ env.JOB_NAME }} Test Results XMLs
          path: ${{ github.workspace }}/**/build/test-results/test/TEST-*.xml
      - uses: actions/upload-artifact@v2
        if: always()
        with:
          name: ${{ env.JOB_NAME }} Test Results HTMLs
          path: ${{ github.workspace }}/**/build/reports/tests/test/
    strategy:
      fail-fast: false
      matrix:
        include:
          - param1: 1
            param2: 2
          - param1: 3
            param2: 4

  publish-test-results:
    needs: check
    if: success() || failure()
    steps:
      - uses: actions/download-artifact@v2
        with:
          path: artifacts

      - name: Publish Test Results
        uses: EnricoMi/publish-unit-test-result-action@v1
        with:
          files: artifacts/* Test Results XMLs/**/*.xml

[mathbunnyru]

I had to manually list all the files to download and it looks really bad. It would be so great to have this feature. https://github.com/jupyter/docker-stacks/blob/7543e49a9fe7f62e95c4b21f96026b2997dfddbc/.github/actions/download-manifests/action.yml

[Andre601]

Just to be sure here, this is about support for downloading multiple uploaded artifacts and not individual files from an artifact right? I want to be sure as I want to make a feature request but want to avoid creating duplicate ones.

[TWiStErRob]

Yep, choose a subset of all artifact ZIP files.

This summarised nicely: https://github.com/actions/download-artifact/issues/6#issuecomment-620634668

Note: either your or this solution might bring us forward, because if there's any way to filter, we don't have to list individual files, nor have to download everything and then filter. The goal is to push filter to server-side, pre-download. Please link your issue as related.

[Andre601]

Yep, choose a subset of all artifact ZIP files.

This summarised nicely: #6 (comment)

Note: either your or this solution might bring us forward, because if there's any way to filter, we don't have to list individual files, nor have to download everything and then filter. The goal is to push filter to server-side, pre-download. Please link your issue as related.

I've opened an issue to suggest the feature I had in mind. https://github.com/actions/download-artifact/issues/176

[PatrickColborne]

I would appreciate this too! I have test coverage reports being generated for a subset of my tests and I need to merge the reports. Each subset runs in its own node and I'm finding it quite challenging to download all the reports and merge them in an elegant way.

[dhruvkb]

Currently, we have worked around it using actions/github-script inside which we use the @actions/artifact package to download the artifacts from a list.

https://github.com/WordPress/openverse/blob/00a26a65e93fe8ea2785d4a26c3ebce2efc935db/.github/actions/load-img/action.yml#L26-L35

It's a hack and requires a bit of work to install the @actions/artifact package so it would be great if the feature was supported natively.

[collin-pasternack]

I would appreciate this too! I have test coverage reports being generated for a subset of my tests and I need to merge the reports. Each subset runs in its own node and I'm finding it quite challenging to download all the reports and merge them in an elegant way.

This is exactly the use case that led me here. I could push my reporting output to a dedicated artifact repository that supports pattern matching and filtering, but then I have to set policies in that tool for a lifetime of < 1 day, whereas the action is a single line.

[Danielku15]

I personally like the TeamCity artifact syntax quite a lot. It gives a nice range of options with wildcards, includes/excludes and alternative names and optional zipping. Having something similar for upload-artifact and download-artifact in GitHub Actions would be awesome.

https://www.jetbrains.com/help/teamcity/configuring-general-settings.html#Artifact+Paths https://www.jetbrains.com/help/teamcity/artifact-dependencies.html#Artifacts+Rules

[per1234]

I think the need for this capability is even greater now that multiple uploads to a single artifact via actions/upload-artifact is no longer allowed:

https://github.com/actions/upload-artifact#breaking-changes

Previously, many of my workflows uploaded to the same artifact multiple times using a job matrix. That single artifact was then downloaded by a dependent job that consumed the output from all the matrix jobs. Due to the change in actions/upload-artifact, it will now be necessary for each matrix job to use a separate artifact. Downloading all those artifacts in the dependent job will be quite messy and difficult to maintain without support for pattern-based downloading of multiple artifacts in the actions/download-artifact action.

[javierjulio]

I wanted to update my usage of actions/upload-artifact and actions/download-artifact to v4 but have run into the same issue as @per1234. I run a matrix of tests by Ruby and Rails versions, that each generate code coverage reports and upload to the same artifact. I can change the upload artifact to be named uniquely based on matrix values but the downloading part is the problem since now I don't see any way other than hardcoding each possible matrix combination to download the artifacts in a job that will then upload the contents of a directory to Codecov.

[hasufell]

Does github have any DevX team that thinks about these things? It doesn't seem so. I'm starting to get Atlassian vibes...

[webknjaz]

Related: https://github.com/actions/upload-artifact/issues/472.

[mathbunnyru]

I suppose the wait is over 🎉 https://github.com/actions/download-artifact/pull/259

v4.1.0 has been released with pattern: feature included. I couldn't be more happy (I also wish v4 download instability issue will be fixed soon as well, then v4 would be awesome).

[vmjoseph]

Closed with the merging of https://github.com/actions/download-artifact/pull/259