azure-devops-yaml-schema icon indicating copy to clipboard operation
azure-devops-yaml-schema copied to clipboard
verified publisher icon MicrosoftDocs

DownloadPipelineArtifact Does Not Appear to Affect `Consumed Artifacts`

Open aolszowka opened this issue 2 years ago • 3 comments

Overview

It appears that the Consumed Artifacts dialog incorrectly reports the artifact consumed when DownloadPipelineArtifact is utilized. It appears to report only the latest artifact from the associated pipeline, as opposed to the actual artifact that is downloaded/consumed.

It is unclear if this is intentional, or if this is a bug, at minimum I feel that the documentation for DownloadPipelineArtifact should be updated to identify this gotcha.

Reproduction / Test Scenario

I have a "Library Pipeline" (lib-pipeline) that I want to have produce artifacts that are then consumed by an "Application Pipeline" (app-pipeline). The design intent is that the app-pipeline will consume resources from the lib-pipeline that are on the same branch. This is leveraged primarily for feature branch development and works extremely well in our environment.

This diagram shows the pipelines and resources:

image

Here are Toy YAML Definitions that help illustrate the problem:

lib-pipeline.yml

steps:
  - download: none
  - task: PowerShell@2
    displayName: 'Create Artifact'
    inputs:
      pwsh: true
      targetType: 'inline'
      script: |
        # Simply Print out the Branch Name From Which This Artifact Comes
        $sourceBranch = '$(Build.SourceBranch)'
        Write-Host $sourceBranch
        $sourceBranch | Out-File '$(Build.ArtifactStagingDirectory)/branchName.txt'
  - task: PublishBuildArtifacts@1
    displayName: 'Publish Artifact'
    inputs:
      PathtoPublish: '$(Build.ArtifactStagingDirectory)'
      ArtifactName: 'drop'
      publishLocation: 'Container'

app-pipeline.yml

trigger: none

resources:
  pipelines:
  - pipeline: lib-pipeline
    source: lib-pipeline
    trigger: true

steps:
  # Disable any automatic download
  - download: none

  # Download only the artifact associated with our branch
  - task: DownloadPipelineArtifact@2
    displayName: 'Download Artifact Associated With Our Branch'
    inputs:
      buildType: 'specific'
      project: $(resources.pipeline.lib-pipeline.projectID)
      definition: $(resources.pipeline.lib-pipeline.pipelineID)
      buildVersionToDownload: 'latestFromBranch'
      branchName: '$(Build.SourceBranch)'

  # Echo out the artifact
  - task: PowerShell@2
    displayName: 'Echo Artifact'
    inputs:
      pwsh: true
      targetType: 'inline'
      script: |
        Get-Content '$(Pipeline.Workspace)/drop/branchName.txt'

It is left as an exercise to the reader to create the pipelines within Azure DevOps named the same as the YAML Files to allow the triggers to work.

The jist of the above is the lib-pipeline simply creates an artifact that contains the name of the branch from which the artifact was created. The app-pipeline attempts to consume the latest artifact, for the branch it was triggered from, and then prints out the value. If the pipeline is correctly configured the echo’ed branch name should be identical to what was triggered on.

Our team is trained to look at this dialog to understand what artifact was consumed, unfortunately this gives incorrect information.

Consider the following run which was triggered off the main branch for app-pipeline, based on the above we would expect the artifact that is consumed to come from the same branch:

image

image

image

In places where we’re not utilizing a feature branch this works great, however in places where we do have a feature branch this is reporting the incorrect artifact as seen above.

In the screenshots above the Azure DevOps Pipeline UI seems to claim that Build 9 was consumed (20230810.6 in the screenshot below), the latest artifact in the lib-pipeline, which happens to come from the feature/123-feature-branch. This is incorrect because the app-pipeline should have consumed the latest main build Build 7 (20230810.5 in the screenshot below).

image

However, if you look at the actual log from the above you will see that it correctly downloaded Build 7 (20230810.5 which was the latest for main).

image

This is further proven in this toy pipeline as the echo of the branch name from which the artifact was created comes back as /ref/heads/main:

image

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

aolszowka avatar Aug 11 '23 16:08 aolszowka