guava icon indicating copy to clipboard operation
guava copied to clipboard

Published 20 hours ago • verified publisher icon google

chore(ci): improvements to gha workflows

Open sgammon opened this issue 1 year ago • 11 comments

Summary

Applies updates to GHA CI, and refactors into Reusable Workflows. Hardens supply chain security for Guava both in CI and release. Fixes and closes #7088.

Related PRs

  • Optional: Included in #7094

Security hardening

  • CI flows now have an enumerated set of allowed internet endpoints. Other endpoint access is blocked.
  • CI flows can now publish SLSA provenance metadata
  • Maven will now reject dependencies which experience hash mismatches
  • Maven dependencies for Guava are now reported to the GitHub Dependency Graph

Enclosed features

Changelog

Expand for full changelog 
- chore(ci): apply hardening to ci jobs
- chore: apply 'Harden Runner' auditing to all ci tasks
- chore: apply `persist-credentials: false` to checkout tasks
- chore: publish dependency graph and add dependency review check
- chore: add codeql scan job (temp)
- chore: bump `actions/checkout` → `4.1.1`
- chore: bump `actions/dependency-review-action` → `4.1.3`
- chore(ci): refactor into reusable workflows
- chore: move ci jobs to `workflow_call` trigger
- chore: add entrypoint jobs for PR and Push events
- chore: cleanup permissions and dispatch checks/tests
- chore(ci): switch to enforced hardening mode
- chore: gather and apply network endpoints for each job
- chore: move to `block` mode for `egress-policy` in `step-security/harden-runner`
- feat(ci): slsa provenance support
- feat: add slsa support to build workflow
- chore: split `test` into `build` and `test` workflows
- chore: use new workflows (build/test) from push/pr triggers
- chore(build): parameterize deploy repositories
- chore: add sigstore plugin to build
- chore: add `--strict-checksums` flag to `mvnw` calls in ci

Action updates

Bumps actions/checkout from 3.6.0 to 4.1.1.

Release notes

Sourced from actions/checkout's releases.

v4.1.1

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v4.1.0...v4.1.1

v4.1.0

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v4.0.0...v4.1.0

v4.0.0

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v3...v4.0.0

Changelog

Sourced from actions/checkout's changelog.

Changelog

v4.1.0

v4.0.0

v3.6.0

v3.5.3

v3.5.2

v3.5.1

v3.5.0

v3.4.0

v3.3.0

v3.2.0

v3.1.0

v3.0.2

... (truncated)

Commits

StepSecurity PR

The PR content below describes the hardening applied by StepSecurity.

Harden Runner

Harden-Runner is an open-source security agent for the GitHub-hosted runner to prevent software supply chain attacks. It prevents exfiltration of credentials, detects tampering of source code during build, and enables running jobs without sudo access.

Harden runner usage

You can find link to view insights and policy recommendation in the build log

Please refer to documentation to find more details.

Detect Vulnerabilities with SAST Workflow

Static Code Analysis (also known as Source Code Analysis) is usually performed as part of a Code Review (also known as clear-box testing) and is carried out at the Implementation phase of a Security Development Lifecycle (SDL). Static Code Analysis commonly refers to the running of Static Code Analysis tools that attempt to highlight possible vulnerabilities within ‘static’ (non-running) source code by using techniques such as Taint Analysis and Data Flow Analysis.

Add Dependency Review Workflow

The Dependency Review Workflow enforces dependency reviews on your pull requests. The action scans for vulnerable versions of dependencies introduced by package version changes in pull requests, and warns you about the associated security vulnerabilities. This gives you better visibility of what's changing in a pull request, and helps prevent vulnerabilities being added to your repository.

sgammon avatar Mar 08 '24 18:03 sgammon

cc / @cpovirk I realize this one is going to look big, but I promise it's worth it if you dig in. Totally happy to close and not merge but I think it will make the build much smoother, and adds some dependency security as well.

sgammon avatar Mar 08 '24 18:03 sgammon

The StepSecurity PR (before cleanup): https://github.com/sgammon/guava/pull/1

Here is a sample run: https://github.com/sgammon/guava/actions/runs/8207681690

sgammon avatar Mar 08 '24 20:03 sgammon

This needs a little bit of cleanup -- mostly an autorebase -- but otherwise it is ready for review @cpovirk

sgammon avatar Mar 11 '24 18:03 sgammon

Okay, excellent, thank you for approving CI workflows by the way. That helps a bunch because I can test running these on an unauthorized fork.

Consequently, the two failures there (SLSA Provenance and CodeQL) are happening because the fork (properly) does not have permissions to id-token: write, security-events: write, etc. Those should be made conditional so that outside fork PRs don't fail but the publishing still works. I'll hunt that down.

sgammon avatar Mar 11 '24 18:03 sgammon

Rebasing this shortly since #7087 was merged

sgammon avatar Mar 12 '24 21:03 sgammon

Provenance/Security stuff

  • Hardening is now strict and denies sudo and network traffic outside of allowlists for each job
  • SLSA provenance is generated on push to master, or on non-fork PRs, but not otherwise
  • SLSA provenance is not published in releases yet, or embedded in the JAR; it can go in a GitHub release, for example
  • SPDX is not published yet (as above for SLSA), but is embedded within the JAR, see JAR tree below (coming soon)
  • Sigstore plugin is added but does not run yet; I don't want to publish signatures of my fork
  • CodeQL is suppressed because "simple" mode must be disabled in the repo before "advanced" mode can be run

The Sigstore plugin is made by Google and Googlers (the infamous @loosebazooka) and it has worked well in the past purely with a GitHub token in CI. I am happy to test it if desired.

Screenshot 2024-03-12 at 3 50 36 PM Screenshot 2024-03-12 at 3 50 57 PM Screenshot 2024-03-12 at 3 53 25 PM

Notes on naming: This SPDX document name and prefix (at META-INF/spdx/) are designed to work well with fat JAR/shadow merging. I'm not aware of standards yet about where to place this document (cc / @goneall @loosebazooka ?)

For instance, maybe this should be put in META-INF/sbom/* so that CycloneDX can be supported side-by-side someday, or other formats, if desired?

SPDX document sample: com.google.guava_guava-HEAD-jre-SNAPSHOT.spdx.json

Multi-JVM testing

Subject to advice on #7087, the build currently maintains the same JVM matrix, and on a subsequent PR I will add multi-JVM testing in one job with Maven toolchains.

Any of these can be rolled back and of course this entire PR is optional for the other changes in flight in #7093 and #7094.

sgammon avatar Mar 12 '24 22:03 sgammon

This SPDX document name and prefix (at META-INF/spdx/) are designed to work well with fat JAR/shadow merging. I'm not aware of standards yet about where to place this document (cc / @goneall @loosebazooka ?)

I recall a discussion in the OpenSSF SBOM Everywhere on creating a best practice guide on where to place SBOMs, but I couldn't find a link to the document/discussion - @joshbressers may be able to provide a pointer.

My personal advice is that the SPDX documents be placed in the directory adjacent to any release artifacts so that it can be picked up by any downstream consumers. If the SPDX document is uploaded to Maven central next to the deployed .jar file, the SPDX maven plugin should recognize it when guava is used in any downstream projects that use the SPDX maven plugin.

BTW - Thanks for adding the SPDX documents to the builds! I use guava in some of my projects, and this will improve the fidelity of the SPDX data I produce for those projects.

goneall avatar Mar 13 '24 01:03 goneall

Just to clarify here, while I am a maintainer of sigstore. It's an openssf/linuxfoundation project. Not a Google project.

loosebazooka avatar May 09 '24 12:05 loosebazooka

Maven central will accept spdx documents as long as they follow the naming convention, and include the necessary signatures/checksums. I don't have any experience with embedding the sbom doc in the jar. I think maven central is a reasonable place to publish sboms (next to the artifact).

Either way I think it's important to make an effort to differentiate dependencies that are shaded in and external dependencies that may or may not change at resolution time. I'm of the opinion that libraries should only include their shaded dependencies in the sbom when publishing to maven central. But I think the jury's still out on this one (@lumjjb, @goneall)

loosebazooka avatar May 09 '24 14:05 loosebazooka

Maven central will accept spdx documents as long as they follow the naming convention, and include the necessary signatures/checksums. I don't have any experience with embedding the sbom doc in the jar. I think maven central is a reasonable place to publish sboms (next to the artifact).

Either way I think it's important to make an effort to differentiate dependencies that are shaded in and external dependencies that may or may not change at resolution time. I'm of the opinion that libraries should only include their shaded dependencies in the sbom when publishing to maven central. But I think the jury's still out on this one (@lumjjb, @goneall)

Dependencies that are included with the distributed artifact(s) can use the CONTAINS relationship. The CONTAINS relationships should not be used for dependencies that are not "contained" in the distributed artifacts. The more general DEPENDENCY_OF relationship can be used for dependencies that may or may not be contained (note: these are not mutually exclusive, you can have both a CONTAINS and DEPENDENCY_OF relationship between the same packages). There are additional dependency relationship types that loosely map to the the Maven scopes.

goneall avatar May 09 '24 17:05 goneall

Gotcha. It sounds like it doesn't make sense to include SBOMs within the JAR itself, so I'll unwind that part and hold it alongside the JAR as an output artifact.

sgammon avatar May 09 '24 20:05 sgammon