awkward

awkward copied to clipboard

• Add generation of GitHub artifact attestations to built sdist and wheel before upload. c.f.:
  • https://github.blog/2024-05-02-introducing-artifact-attestations-now-in-public-beta/
  • https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds
• Add verification of artifact attestation before publishing awkward to PyPI using the 'gh attestation verify' CLI API, added in v2.49.0.
  • c.f. https://github.com/cli/cli/releases/tag/v2.49.0

As an example c.f. https://github.com/scikit-hep/pyhf/pull/2473.

Debug examples running on my fork:

• Example for deploying awkward-cpp

Attestation created for awkward-cpp-33.tar.gz@sha256:ac86c8e77e024afc75cb16d84fab16a95b343183ac47017f332ee35b64553abe
Attestation signed using certificate from Public Good Sigstore instance
Attestation signature uploaded to Rekor transparency log
https://search.sigstore.dev?logIndex=95705050
Attestation uploaded to repository
https://github.com/matthewfeickert/awkward/attestations/884514

• Example for deploying awkward

Attestation created for awkward-2.6.4-py3-none-any.whl@sha256:c9dc7d953f9c48cad502543176ccc9db2634a6b52dfdcf1fd93b9a1337fbe621
Attestation signed using certificate from Public Good Sigstore instance
Attestation signature uploaded to Rekor transparency log
https://search.sigstore.dev?logIndex=95694782
Attestation uploaded to repository
https://github.com/matthewfeickert/awkward/attestations/883802
Attestation created for awkward-2.6.4.tar.gz@sha256:8b95026cda9814b64bdf769c0af0c9b72a9e3d77967d45c02b78aa883a8eae51
Attestation signed using certificate from Public Good Sigstore instance
Attestation signature uploaded to Rekor transparency log
https://search.sigstore.dev?logIndex=95694788
Attestation uploaded to repository
https://github.com/matthewfeickert/awkward/attestations/883803

Run gh attestation verify dist/awkward-*.tar.gz --repo matthewfeickert/awkward
matthewfeickert/awkward	https://slsa.dev/provenance/v1	.github/workflows/deploy.yml@refs/heads/debug/add-artifact-attestations
Run gh attestation verify dist/awkward-*.whl --repo matthewfeickert/awkward
matthewfeickert/awkward	https://slsa.dev/provenance/v1	.github/workflows/deploy.yml@refs/heads/debug/add-artifact-attestations

These debug examples are running on push events on a debug branch I made, but this PR implements things so that they only create attestations when the publishing workflow runs (so on workflow dispatch or release events).

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 82.11%. Comparing base (b749e49) to head (8f01e17). Report is 90 commits behind head on main.

Additional details and impacted files

see 85 files with indirect coverage changes

This is fantastic! Thanks all :)

Nice to see for awkward-cpp v34 and awkward v2.6.5: https://github.com/scikit-hep/awkward/attestations

$ python -m pip download --no-deps awkward
Collecting awkward
  Downloading awkward-2.6.5-py3-none-any.whl.metadata (7.0 kB)
Downloading awkward-2.6.5-py3-none-any.whl (796 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 796.1/796.1 kB 5.3 MB/s eta 0:00:00
Saved ./awkward-2.6.5-py3-none-any.whl
Successfully downloaded awkward
$ gh attestation verify awkward-2.6.5-py3-none-any.whl --repo scikit-hep/awkward
Loaded digest sha256:9fdecd255b660e05369631331689ca6ffedc1047e7756aa402445b4fa96081ce for file://awkward-2.6.5-py3-none-any.whl
Loaded 1 attestation from GitHub API
✓ Verification succeeded!

sha256:9fdecd255b660e05369631331689ca6ffedc1047e7756aa402445b4fa96081ce was attested by:
REPO                PREDICATE_TYPE                  WORKFLOW                                     
scikit-hep/awkward  https://slsa.dev/provenance/v1  .github/workflows/deploy.yml@refs/tags/v2.6.5

Though I messed up with

https://github.com/scikit-hep/awkward/blob/45c708d29db368eaf451ff7b323b32bcebbef46c/.github/workflows/deploy-cpp.yml#L37-L40

as the naming scheme is awkward-cpp-*.tar.gz and awkward_cpp-*.whl so only the sdist for awkward-cpp got signed

# $ python -m pip download --no-binary :all: --no-deps awkward-cpp  # isn't working to get the sdist without build?
$ curl -sLO https://files.pythonhosted.org/packages/bd/0d/64ce9e9f6164a1bf8f0c32f86f5e705598093f16158d9b73b3e33c00ec35/awkward-cpp-34.tar.gz
$ gh attestation verify awkward-cpp-34*.tar.gz --repo scikit-hep/awkward
Loaded digest sha256:39ac814d769f38ade9a5f6c39b85b07bbc05af56342fb15742edce1237f9e049 for file://awkward-cpp-34.tar.gz
Loaded 1 attestation from GitHub API
✓ Verification succeeded!

sha256:39ac814d769f38ade9a5f6c39b85b07bbc05af56342fb15742edce1237f9e049 was attested by:
REPO                PREDICATE_TYPE                  WORKFLOW                                        
scikit-hep/awkward  https://slsa.dev/provenance/v1  .github/workflows/deploy-cpp.yml@refs/heads/main
$ python -m pip download --no-cache-dir --no-deps awkward-cpp
Collecting awkward-cpp
  Downloading awkward_cpp-34-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.2 kB)
Downloading awkward_cpp-34-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (694 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 694.1/694.1 kB 6.0 MB/s eta 0:00:00
Saved ./awkward_cpp-34-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Successfully downloaded awkward-cpp
$ gh attestation verify awkward_cpp-34*.whl --repo scikit-hep/awkward
Loaded digest sha256:077f6ae40e2a4fdda1252665eeebb7ab52d3987924e861fe5c8f97b473b540b5 for file://awkward_cpp-34-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
✗ Loading attestations from GitHub API failed

Error: failed to fetch attestations from scikit-hep/awkward: HTTP 404: Not Found (https://api.github.com/repos/scikit-hep/awkward/attestations/sha256:077f6ae40e2a4fdda1252665eeebb7ab52d3987924e861fe5c8f97b473b540b5?per_page=30)