DataflowTemplates icon indicating copy to clipboard operation
DataflowTemplates copied to clipboard

Generate SBOM and Provenance for released images

Open Polber opened this issue 1 year ago • 2 comments

PR adds SBOM and Provenance generation for released images, as well as Build time scanning for these images.

Notable changes are:

  • Building images with Provenance (more info here.)
    • Images built with JIB (Java-only flex templates) are now packed as tar file instead of being pushed to GCR directly. These tar files are then sent to Cloud Build to be loaded (docker load) and pushed to GCR/AR - This takes advantage of the built-in Provenance support that Cloud Build provides for all images pushed to GCR/AR
    • Images built solely with Cloud Build (Python, Xlang and YAML templates) will still be built using the Kaniko executor, but will likewise be packaged as tar before being sent to next step where they are loaded (docker load) and pushed to GCR/AR.
  • Generating SBOM
    • SBOM generation is invoked on the image(s) after being pushed to GCR/AR using the command gcloud artifacts sbom export --uri=gcr.io/.... More info about SBOM generation can be found here.
  • Build-time scanning
    • ALL images will now be sent to Cloud Build as described in the Provenance section above. This not only allows us to leverage the Provenance generation, but also Build-time scanning. All the released images will be stored in a repository with continuous scanning, but this build-time scan allows critical vulnerabilities to be found before release, to hopefully avoid vulnerabilities earlier in the lifecycle of a released image.
    • This will use a separate Cloud Build job that runs after the image is staged. This job takes the released image as an argument and runs the scan against it.

All the above changes will be behind a maven argument -DgenerateSBOM which is disabled by default when Staging and Running a template (i.e. for ITs), but is enabled by default for Release.

Polber avatar Sep 21 '24 15:09 Polber

Codecov Report

Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Project coverage is 45.10%. Comparing base (515e05f) to head (923791e). Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...gle/cloud/teleport/plugin/TemplatePluginUtils.java 0.00% 2 Missing :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1886      +/-   ##
============================================
+ Coverage     45.09%   45.10%   +0.01%     
+ Complexity     3614     3612       -2     
============================================
  Files           835      835              
  Lines         49720    49739      +19     
  Branches       5237     5240       +3     
============================================
+ Hits          22421    22435      +14     
- Misses        25633    25635       +2     
- Partials       1666     1669       +3     
Components Coverage Δ
spanner-templates 66.24% <ø> (-0.02%) :arrow_down:
spanner-import-export 64.16% <ø> (-0.07%) :arrow_down:
spanner-live-forward-migration 76.11% <ø> (ø)
spanner-live-reverse-replication 76.48% <ø> (ø)
spanner-bulk-migration 85.50% <ø> (ø)
Files with missing lines Coverage Δ
...gle/cloud/teleport/plugin/TemplatePluginUtils.java 0.00% <0.00%> (ø)

... and 3 files with indirect coverage changes

codecov[bot] avatar Sep 21 '24 16:09 codecov[bot]

Test failure looks unrelated

Polber avatar Oct 14 '24 16:10 Polber