DataflowTemplates
DataflowTemplates copied to clipboard
Generate SBOM and Provenance for released images
PR adds SBOM and Provenance generation for released images, as well as Build time scanning for these images.
Notable changes are:
- Building images with Provenance (more info here.)
- Images built with JIB (Java-only flex templates) are now packed as tar file instead of being pushed to GCR directly. These tar files are then sent to Cloud Build to be loaded (
docker load) and pushed to GCR/AR - This takes advantage of the built-in Provenance support that Cloud Build provides for all images pushed to GCR/AR - Images built solely with Cloud Build (Python, Xlang and YAML templates) will still be built using the Kaniko executor, but will likewise be packaged as tar before being sent to next step where they are loaded (
docker load) and pushed to GCR/AR.
- Images built with JIB (Java-only flex templates) are now packed as tar file instead of being pushed to GCR directly. These tar files are then sent to Cloud Build to be loaded (
- Generating SBOM
- SBOM generation is invoked on the image(s) after being pushed to GCR/AR using the command
gcloud artifacts sbom export --uri=gcr.io/.... More info about SBOM generation can be found here.
- SBOM generation is invoked on the image(s) after being pushed to GCR/AR using the command
- Build-time scanning
- ALL images will now be sent to Cloud Build as described in the Provenance section above. This not only allows us to leverage the Provenance generation, but also Build-time scanning. All the released images will be stored in a repository with continuous scanning, but this build-time scan allows critical vulnerabilities to be found before release, to hopefully avoid vulnerabilities earlier in the lifecycle of a released image.
- This will use a separate Cloud Build job that runs after the image is staged. This job takes the released image as an argument and runs the scan against it.
All the above changes will be behind a maven argument -DgenerateSBOM which is disabled by default when Staging and Running a template (i.e. for ITs), but is enabled by default for Release.
Codecov Report
Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
Project coverage is 45.10%. Comparing base (
515e05f) to head (923791e). Report is 2 commits behind head on main.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| ...gle/cloud/teleport/plugin/TemplatePluginUtils.java | 0.00% | 2 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #1886 +/- ##
============================================
+ Coverage 45.09% 45.10% +0.01%
+ Complexity 3614 3612 -2
============================================
Files 835 835
Lines 49720 49739 +19
Branches 5237 5240 +3
============================================
+ Hits 22421 22435 +14
- Misses 25633 25635 +2
- Partials 1666 1669 +3
| Components | Coverage Δ | |
|---|---|---|
| spanner-templates | 66.24% <ø> (-0.02%) |
:arrow_down: |
| spanner-import-export | 64.16% <ø> (-0.07%) |
:arrow_down: |
| spanner-live-forward-migration | 76.11% <ø> (ø) |
|
| spanner-live-reverse-replication | 76.48% <ø> (ø) |
|
| spanner-bulk-migration | 85.50% <ø> (ø) |
| Files with missing lines | Coverage Δ | |
|---|---|---|
| ...gle/cloud/teleport/plugin/TemplatePluginUtils.java | 0.00% <0.00%> (ø) |
Test failure looks unrelated