scorecard icon indicating copy to clipboard operation
scorecard copied to clipboard

BUG `.sigstore` bundles are not being found by Signed-Releases check

Open cpswan opened this issue 1 year ago • 3 comments

Describe the bug The Signed-Releases score is 0 even when .sigstore bundles are present for the last 5 releases.

Reproduction steps Steps to reproduce the behavior:

  1. Create a repo that signs distribution artifacts with the sigstore/gh-action-sigstore-python GitHub Action
  2. Add a scorecard to that repo
  3. Publish a few releases (and confirm that they get a bunch of .sigstore bundles)
  4. Observe that the Signed-Releases score is still 0

Expected behavior Signed-Releases score should be 8 after 5 releases with .sigstore bundles.

Additional context This was initially raised by @edgarrmondragon in #3771 and should have been fixed by #3772

I first noticed this with the at_python repo I maintain:

{
  "name": "Signed-Releases",
  "score": 0,
  "reason": "0 out of 5 artifacts are signed or have provenance",
  "details": [
    "Warn: release artifact v0.2.6 does not have provenance: https://api.github.com/repos/atsign-foundation/at_python/releases/143607745",
    "Warn: release artifact v0.2.6 not signed: https://api.github.com/repos/atsign-foundation/at_python/releases/143607745",
    "Warn: release artifact v0.2.5 does not have provenance: https://api.github.com/repos/atsign-foundation/at_python/releases/143063709",
    "Warn: release artifact v0.2.5 not signed: https://api.github.com/repos/atsign-foundation/at_python/releases/143063709",
    "Warn: release artifact v0.2.4 does not have provenance: https://api.github.com/repos/atsign-foundation/at_python/releases/142574958",
    "Warn: release artifact v0.2.4 not signed: https://api.github.com/repos/atsign-foundation/at_python/releases/142574958",
    "Warn: release artifact v0.2.3 does not have provenance: https://api.github.com/repos/atsign-foundation/at_python/releases/140371553",
    "Warn: release artifact v0.2.3 not signed: https://api.github.com/repos/atsign-foundation/at_python/releases/140371553",
    "Warn: release artifact v0.2.2 does not have provenance: https://api.github.com/repos/atsign-foundation/at_python/releases/138635865",
    "Warn: release artifact v0.2.2 not signed: https://api.github.com/repos/atsign-foundation/at_python/releases/138635865"
  ],
  "documentation": {
    "short": "Determines if the project cryptographically signs release artifacts.",
    "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#signed-releases"
  }
}

Here's the latest release assets using gh release view --json assets -q .assets.[].name

atsdk-0.2.7-py3-none-any.whl
atsdk-0.2.7-py3-none-any.whl.sigstore
atsdk-0.2.7.tar.gz
atsdk-0.2.7.tar.gz.sigstore

Previous releases are the same aside from the semver bumps.

As @edgarrmondragon previously raised this I took a look at his citric and the same is happening there. The .sigstore bundles are present, but the Signed-Releases score is 0.

cpswan avatar Mar 04 '24 17:03 cpswan

I think the issue is just that there hasn't been a new release in a while. After that, the scorecard-action Docker image should be updated.

edgarrmondragon avatar Mar 04 '24 18:03 edgarrmondragon

@edgarrmondragon now that I look... 🤦

I'm so accustomed to 'release early, release often' (and so used to bumping the Scorecard workflow for CodeQL changes) I'd missed that 5 months have rolled by without a release.

cpswan avatar Mar 04 '24 18:03 cpswan

We've been working on a major release, and haven't released in a while as we work through some breaking changes. I can try to see how many breaking changes have slipped in, and if an interim release is possible.

spencerschrock avatar Mar 04 '24 20:03 spencerschrock

Cutting a Scorecard release may be challenging at this point before our v5 release within the month.

Would it be sufficient for us to update Scorecard Action to use a newer version of scorecard (via a Go pseudo-version)

spencerschrock avatar Mar 25 '24 18:03 spencerschrock

Would it be sufficient for us to update Scorecard Action to use a newer version of scorecard (via a Go pseudo-version)

Yeah, I guess most folks are using scorecard via ossf/scorecard-action. It'd at least work for me.

edgarrmondragon avatar Mar 25 '24 20:03 edgarrmondragon

The fix has now been included in both a ossf/scorecard release (v5.0.0-rc1 and v5.0.0-rc2) and a ossf/scorecard-action release (v2.3.3).

spencerschrock avatar May 09 '24 21:05 spencerschrock