Download from a different workflow

[mastoj]

How do you download an artifact from another workflow?

A little bit of background. We have a build workflow that builds an artifact that we publish. Then we have a separate workflow that is triggered on deployment that needs to access this artifact. I tried to just use the same name for the artifact in download-artifact and upload-artifact hoping that it would download the files, that doesn't seem to work.

[mastoj]

It is a little bit unclear about the direction that you intended here. I definitely see some workarounds:

• Create a release and use the sha as tag. Don't like this approach since this will create a release of every build and mess up the releases.
• Use the package registry maybe? Can't do this right now since we don't have access

Maybe we are doing something completely wrong, and that is fine since it is quite new :). With that said, if we can't access the artifacts we have to revert to circle ci for the build and use github actions only for deploy.

[TingluoHuang]

@chrispat for feedback

[kimble]

This could also be relevant for build caching.

[cormacrelf]

And also for diffing artifacts between master and a pull request. There are hundreds of apps that do this for visual/webpage diffs, but none I can find that can diff a simple text file artifact.

[zlepper]

In addition to this: How to do this cross repositories?

Also we would like to limit the download to certain branches (Using somewhat the same logic as what TeamCity uses with their artifact dependencies), for example we can use the build output of one repository master, in another repository (creating a master build from several smaller projects). Preferably this would use the same branch if possible, or fallback to the default branch if possible.

[sjackman]

I'm very excited for this feature. I want to build and upload artifacts from a fork PR, and this part works well already. Fork PRs run without access to secrets, so in a second GitHub Action workflow that runs with access to secrets (outside of the fork PR), I want to download artifacts from GitHub and upload them to another service (BinTray). I had this arrangement working using CircleCI. This missing feature is blocking me from migrating from CircleCI to GitHub Actions, so I'm very keen to see it implemented!

[sjackman]

H, @ethomson! I work with @MikeMcQuaid on Homebrew. He told me that you may be a good person to give this issue some 👀. See also the related issue https://github.com/actions/upload-artifact/issues/21#issuecomment-562662657.

[mikkelbd]

Same use case as OP here. Any progress on this? 🙏 It would be really helpful to access an artifact uploaded from a build-workflow in a deploy-workflow. Especially since the cache action doesn't support deployment events: https://github.com/actions/cache/issues/63. Otherwise the cache could probably have been used with e.g. "artifact-name-$GITHUB_SHA" even though it's kind of a hack.

[chrispat]

@mikkelbd we now have an api for downloading artifactshttps://developer.github.com/v3/actions/artifacts/#download-an-artifact so you could likely accomplish your scenario now.

[eine]

@chrispat, I believe that @mikkelbd, as me and probably many others, is expecting this action to support the feature. In the end, if we are all forced to write custom scripts that interact with the API, all the "Actions" infrastructure is useless. This is specially unfortunate because GitHub's CI service as a product seems to be severely biased towards Actions.

[mikkelbd]

@chrispat Thank you for the prompt response :clap: Artifacts being available in the API is a good start. Combined with https://github.com/octokit/request-action or https://github.com/actions/github-script one could probably download it without writing custom code. However, it seems that you would have to know the internal artifact_id of the artifact, or you would have to list all artifacts and then filter by name? Again, then you would have to know the run_id that uploaded the artifact? I guess that's pretty hard in another workflow run...

As @eine stated, it would be a much better experience if the download-artifact action supported this out of the box instead of forcing n developers to interact with the GitHub API and also making the workflow-file bloated with low-level stuff 😅

[sjackman]

we now have an api for downloading artifactshttps://developer.github.com/v3/actions/artifacts/#download-an-artifact so you could likely accomplish your scenario now.

@chrispat Is there an API to get all the workflow runs for a given pull request or commit SHA-1? I found this API list-workflow-runs to list all the runs for a given repo and workflow, and that could then be filtered to find the SHA1 of the commit that you're interested in, but not terribly efficient.

https://developer.github.com/v3/actions/workflow_runs/#list-workflow-runs

[chrispat]

We don't currently have that parameter to the API but I will add it to our feedback list to consider for the future.

[sjackman]

pr="$(echo '${{github.event.head_commit.message}}' | sed 's/^.*#\([0-9]*\).*/\1/;q')"
git -C "$(brew --repo ${{github.repository}})" fetch origin "pull/$pr/head:pr"
sha1="$(git -C "$(brew --repo ${{github.repository}})" rev-parse pr)"
echo pr="$pr" sha1="$sha1"
run_id=$(curl -s -H 'Accept: application/vnd.github.antiope-preview+json' https://api.github.com/repos/${{github.repository}}/actions/workflows/build-bottles.yml/runs \
  | jq ".workflow_runs[] | select(.head_sha == \"$sha1\").id")
artifact_id="$(curl -s -H 'Accept: application/vnd.github.antiope-preview+json' https://api.github.com/repos/${{github.repository}}/actions/runs/$run_id/artifacts \
  | jq '.artifacts[0].id')"
echo run_id="$run_id" artifact_id="$artifact_id"
curl -L -o bottles.zip "https://${{secrets.GITHUB_PAT}}@api.github.com/repos/${{github.repository}}/actions/artifacts/$artifact_id/zip"

https://github.com/brewsci/homebrew-bio/blob/master/.github/workflows/upload-bottles.yml I've successfully used the new workflows and artifacts endpoints to download the artifact of given PR number in a push event! Yeah! Any comments or suggestions to make this easier would be welcome.

Are artifacts of a public repo public, and why is a GITHUB_PAT needed to download an artifact? GITHUB_TOKEN wasn't sufficient.

[jpfeuffer]

I found the same oddity. It seems like downloading artifacts requires "admin" permissions instead of "read" permissions (as documented).

It would really be great if this would be a standard feature of this official GitHub action. Is there any progress on this?

[josejulio]

I found the same oddity. It seems like downloading artifacts requires "admin" permissions instead of "read" permissions (as documented).

Is this reported somewhere? If not, where would be the ideal place to report it?

[davidobrien1985]

It would be great if we'd be able to access builds from other repos in our org through this action. Is that planned at all?

[josejulio]

It would be great if we'd be able to access builds from other repos in our org through this action. Is that planned at all?

This action doesn't do that yet, but you can do it with curl

see: https://github.com/RedHatInsights/policies-ui-frontend/blob/master/.github/scripts/download-latest-openapi.sh#L23

You need to pass the token in the workflow for it to work correctly: https://github.com/RedHatInsights/policies-ui-frontend/blob/master/.github/workflows/OpenAPIInSync.yml#L24

[mk-nickyang]

Here's an example of downloading artifact from a different workflow using github-script, hopefully it can help someone https://gist.github.com/mk-nickyang/c8094f6b32472eaebdd3c21b1ccb7e86

[fkirc]

I also believe that download-artifact is too restrictive, because sharing artifacts within a single workflow-run is not the only use case of Artifacts.

I am aware that there exists an artifacts-API that is separate from download-artifact. However, this API seems to lack name-addressable artifacts. Instead, this API requires some internal artifact_id. To get such an artifact_id, one has to either fetch a list of all artifacts in a repo, or fetch artifacts via the workflow-runs-API. But then again, it is painful to find id's for the workflow-runs-API.

To recap, all I want is to download an artifact-file with a specific name, where the artifact-file originates from a different workflow-run (either from the same or a different workflow file, this should not matter). The actions-API seems way too convoluted for this purpose; not to mention that this API uses ZIP-files and enforces actions to use secrets.GITHUB_TOKEN.

Personally, I won't bother to re-invent the wheel before something like https://github.com/actions/toolkit/pull/517 gets merged. In the meantime, I recommend to use services other than GitHub for artifact storage and deployment.

[josejulio]

I use this one to download artifacts from other repository: https://github.com/dawidd6/action-download-artifact Before that I used this script: https://github.com/RedHatInsights/policies-ui-frontend/blob/03870393c706db5d4f30d3708058bb3dd85241a2/.github/scripts/download-latest-openapi.sh

[fkirc]

Thanks for the hint, I will give it a try. For medium-sized projects, the scalability and reliability problems of the current solutions will hopefully not matter (think of large downloads or a huge number of Action-runs).

[baratgabor]

I'd also add my two cents in support of this feature. I hardly know anything about CI/CD, because I'm used to leads having that responsibility, but on my own project the very first thing I tried to do is to:

• build, test, and upload the tested build artifact in a build workflow (triggered by pull request on main), and then
• download the same artifact in a separate publish workflow, and publish it (triggered by push on main).

And I was surprised to find out that there is totally no built-in support for this, because I assumed this is a common thing to do.

[altendky]

@baratgabor, fwiw, I just put such jobs at the end of a single workflow and use needs: so they only run if everything else succeeds. Not the same, different tradeoffs, but you don't have to wait for this to get implemented.

[cedric05]

how do i down artifact for integration testing. it seems user has to be logged in for artifacts to be downloadable (for public repository)

artifacts:

• downloadable for logged in user releases:
• downloadable for anonymous user

Will this use case be covered, should i look for alternatives?

https://github.com/cedric05/dothttp-runner/blob/753fb1ab46b537d3c63027e0dc62e6090771a79e/version.json#L48-L56

[StephenCleary]

Being able to specify artifacts by run_id and name would make this common scenario much easier: https://securitylab.github.com/research/github-actions-preventing-pwn-requests/

Having to copy/paste the github-script everywhere isn't that great of a user experience, since most open-source repositories would benefit from the pull_request + workflow_run combo.

[oppknox]

We are at 606 days... does anyone work here?

[eine]

@oppknox see https://github.com/actions/upload-release-asset/issues/58.

[tiliv]

I'm leaving a +1 to be annoying. I'm shocked that downloading by workflow name has to be hand rolled by every single user when there's a perfectly good action right here.

[oppknox]

@tiliv Hey Autumn, I'm still annoyed that they have completely abandoned the community, but luckily I finally found a solution that ended up working for me. Praise be to dawidd6.

I highly recommend https://github.com/dawidd6/action-download-artifact. This paired well with a workflow dispatch job using RunId or if null use Latest/Success.