azuredevops icon indicating copy to clipboard operation
azuredevops copied to clipboard

Unable to upload artifact of multiple task executions with multiple report formats

Open emboss64 opened this issue 2 years ago • 10 comments

When you run dependency-check-build-task multiple times within the same pipeline your artifact only stores the files from the last run. This wouldn't be an issue if you were only using a single output format and using reportFilename, which is not our case as we need both JUnit and HTML reports.

As a workaround, we are uploading artifacts after every run modifying the filename, but this results in also having the original filename

        write-Host "##vso[task.addattachment type=dependencycheck-artifact;name=$new_name;]$new_path"
        write-Host "##vso[artifact.upload containerfolder=dependency-check;artifactname=Dependency Check;]$new_path"

image

Some options I can think of to fix this:

  • Allowing to override the containerfolder or artifactname with an extra parameter in: https://github.com/dependency-check/azuredevops/blob/c4e46d015524bc5ce61361813c118076f7a8e13b/src/Tasks/dependency-check-build-task/dependency-check-build-task.ts#L218
  • Keep the same containerfolder and artifactname but allow prefixes or suffixes for artifact filenames.
  • Flag to disable artifact upload (and then manually uploading files/artifacts via traditional methods if needed)

emboss64 avatar Feb 24 '23 17:02 emboss64

Hello, may I ask you why it's needed to execute it multiple times in the same pipeline?

Following is not tested.

Maybe I got your scenario wrong, but you can get the output formats JUnit and HTML with one execution. Define them as list comma separated for instance JUnit,HTML as the value of parameter format:

  • format: JUnit,HTML

In addition, for multiple runs within the the same pipeline, you can define the path to your artifacts drop-off root directory and a sub-folder for each run by using the parameter reportDirectory.

  • run # 1
    • reportFormat: $(Build.ArtifactStagingDirectory)/dependency-check/run-1
  • run # 2
    • reportFormat: $(Build.ArtifactStagingDirectory)/dependency-check/run-2
  • run # 3
    • reportFormat: $(Build.ArtifactStagingDirectory)/dependency-check/run-3

HaGGi13 avatar Feb 24 '23 18:02 HaGGi13

Hi! So basically we've predefined templates for mono repo solutions which mix different tech stacks. So the same template is executed twice within the same pipeline but with a type switch which enables/disables analyzers based on the tech stack.

I'm going to give the reportDirectory parameter a go, to see if that actually does the trick! Thanks! I thought that having containerfolder=dependency-check in dependency-check-build-task.ts would always end up setting the artifacts in the same container within the published Dependency Check artifact

emboss64 avatar Feb 26 '23 21:02 emboss64

I can confirm it does not work. The reports are generated indeed in a separate directory but at the time of uploading the artifact they are uploaded to the same target containerfolder

Some logs from a sample run

##[debug]reportsDirectory=/opt/tfs/agent01/_work/341/a/dependency-check/net/
Setting report directory to /opt/tfs/agent01/_work/341/a/dependency-check/net/
##[debug]Absolute path for pathSegments: /opt/tfs/agent01/_work/341/a/dependency-check/net/,log = /opt/tfs/agent01/_work/341/a/dependency-check/net/log
Creating report directory at /opt/tfs/agent01/_work/341/a/dependency-check/net/

##[debug]Upload artifact: /opt/tfs/agent01/_work/341/a/dependency-check/net/dependency-check-junit.xml to server for build: 506609 at backend.
##[debug]Processed: ##vso[artifact.upload containerfolder=dependency-check;artifactname=Dependency Check;]/opt/tfs/agent01/_work/341/a/dependency-check/net/dependency-check-junit.xml

emboss64 avatar Feb 27 '23 09:02 emboss64

I have the same problem

pippolino avatar Nov 01 '23 16:11 pippolino

@ejohn20 @guidojw @HaGGi13 Thank you for all the support you give to this repository, I have seen that no interventions have been made for many months. Is the project always maintained?

pippolino avatar Nov 06 '23 09:11 pippolino

We are no longer are using Azure DevOps in our organization, so I have not been actively maintaining this. I'm open to having other folks take over this extension if the desire is there. @jeremylong is there an official way you have done this in the past?

ejohn20 avatar Nov 07 '23 06:11 ejohn20

If a new volunteer to work on the project steps forward - we can give them access. This has happened in the past with the ODC jenkins plugin. However, I do not have time myself to maintain the ADO extension myself.

jeremylong avatar Nov 07 '23 11:11 jeremylong

@ejohn20 I might suggest that you update the README.md indicating the project is seeking a new maintainer?

jeremylong avatar Nov 07 '23 11:11 jeremylong

@jeremylong I am available to help if necessary

pippolino avatar Nov 07 '23 13:11 pippolino

READMD updated and I opened an issue specifically for this: https://github.com/dependency-check/azuredevops/issues/137

ejohn20 avatar Nov 08 '23 16:11 ejohn20