python
python copied to clipboard
docker-library
PEP 761: Deprecating PGP signatures
Hello, I noticed this project uses PGP signatures and wanted to make you all aware of my proposed PEP for deprecating PGP signatures in future Python versions. Please take a look and leave questions in the discussion thread.
Oof, not ideal, but something we can/will adapt to if it's adopted.
Thank you so much for the proactive heads up! It's really, really appreciated. :heart:
The more I look at the details, the more concerned I am about this -- sigstore clients are not super generally available (doesn't appear to have pre-existing Debian or Alpine packages, for example, and definitely not in bookworm/stable), especially before we've built/installed Python (which we need to verify before we do :grimacing:), but even if it weren't, installing new packages from PyPI during our build is not something we're comfortable with (we just got away from installing pip from PyPI :sweat_smile:). :thinking:
To put that another way, this creates a new verification problem for us in that we need to somehow fetch/verify/trust a sigstore client as well. :disappointed:
We'll also need to maintain a mapping of release to release manager + OIDC Issuer (https://www.python.org/downloads/metadata/sigstore/), which is similar to what we already maintain for PGP, so not a huge burden but definitely not easier for us. :sweat_smile: (Already addressed explicitly in https://discuss.python.org/t/pep-761-deprecating-pgp-signatures-for-cpython-artifacts/67180/6 though, which is nice :heart:)
(Some variant of this might belong in the upstream discussion thread, but I'm mostly trying to write down ideation about how we would implement this, so for now I'm keeping it local. I do not mind if I get quoted in the upstream thread if something here is useful there, though -- I'm "tianon" there too if that happens and the quoter wants to @ me.)
We could use sigstore-go, as suggested by the upstream documentation, but we'd have to compile it ourselves because (somewhat ironically) that project doesn't publish precompiled binaries (and solving the "verification" problem there is an interesting/fun case of chicken-and-egg).
As an affected downstream, it would honestly be more useful to us to have a published checksum that's stronger than MD5 and more machine-readable than what's listed in the HTML table on https://www.python.org/downloads/release/python-3130/ (I don't think that value lives anywhere under https://www.python.org/ftp/python/3.13.0/, for example, that we could easily scrape it from).
(Yes, checksum and signature verify different things -- however, the different things they verify are valuable separately, and the checksum is a stronger verification for our purposes than the signatures are.)
@tianon That's good to know that stronger checksum value would be useful, there's an open issue for it and I plan to tackle it myself (upgrading to SHA256). Also: a feature that's different than PGP is that Sigstore can verify a checksum value and signature against an identity (rather than requiring the original binary blob).
Very nice, that would be awesome! I dug a little more and found that the SPDX files published do include a checksum of the published artifacts too, but it's really buried so it'll require some finagling to extract. :sweat_smile:
$ wget -qO- 'https://www.python.org/ftp/python/3.13.0/python-3.13.0.exe.spdx.json' | jq '.packages[] | select(.packageFileName == "python-3.13.0.exe")'
{
"SPDXID": "SPDXRef-PACKAGE-cpython",
"checksums": [
{
"algorithm": "SHA256",
"checksumValue": "a9be7082ccd3d0b947d14a87bceadb1a3551382a68fcb64d245a2ebcc779b272"
}
],
"downloadLocation": "https://www.python.org/ftp/python/3.13.0/python-3.13.0.exe",
"externalRefs": [
{
"referenceCategory": "SECURITY",
"referenceLocator": "cpe:2.3:a:python:python:3.13.0:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
}
],
"licenseConcluded": "PSF-2.0",
"name": "CPython",
"originator": "Organization: Python Software Foundation",
"packageFileName": "python-3.13.0.exe",
"primaryPackagePurpose": "LIBRARY",
"supplier": "Organization: Python Software Foundation",
"versionInfo": "3.13.0"
}
Also: a feature that's different than PGP is that Sigstore can verify a checksum value and signature against an identity (rather than requiring the original binary blob).
Exploration of this thought gave me a simpler solution:
$ wget -qO- 'https://www.python.org/ftp/python/3.10.15/Python-3.10.15.tar.xz.sigstore' | jq -r '.messageSignature.messageDigest.digest' | base64 -d | hexdump -ve '/1 "%02x"'; echo
aab0950817735172601879872d937c1e4928a57c409ae02369ec3d91dccebe79
(extracting the SHA256 straight from the sigstore bundles, which lets me get values further back than the 3.12 that added SPDX files :eyes:)
... SHA256 verification based on that is now implemented in https://github.com/docker-library/python/pull/978 :smile:
To reiterate for the avoidance of doubt, I don't think this solves the same problem as actually verifying the provenance of the builds too, but it does make it feel less urgent for us to find a good solution to the chicken-and-egg problem that getting a suitable sigstore client during our builds is (and ultimately we'd like to be verifying both).
Gave sigstore verification a first shot in #983
- a suitable cosign version is only available in Alpine Edge
- cosign is not packaged for Debian at all
- verification using the python sigstore client doesn't seem like a great option either
Only other option would be to do the download/check in a separate stage first (not pretty).
Indeed, getting an appropriate sigstore client here (either one of the direct sigstore options or cosign) in a secure and verifiable way is going to be our biggest hurdle, especially since none of them are packaged in Debian yet (and it'd be years before we'd see them in a stable release even if we start that work today).
We could download and/or build one of them from source in our Dockerfile, but then we have the same chicken-and-egg problem with verifying those, and if we're still trusting them with just a checksum, then we might as well only use the checksums from Python (now that we've got them, thanks to #978).
As noted in https://github.com/docker-library/python/pull/983#issuecomment-2423244192, it's great that Alpine Edge has cosign, but that unfortunately won't help us until Alpine 3.21 (and that's assuming it does successfully land in 3.21 - packages in Edge often don't). Additionally, as discussed in https://github.com/docker-library/python/pull/980#issuecomment-2420843631, I've been looking at https://github.com/python/cpython/issues/90548 again recently and the fact that Alpine (slash musl) isn't actually a supported upstream platform, and contemplating what we should do about our Alpine builds in the future, with not maintaining them anymore being one option that's on the table.
To put that more succinctly, I'm currently questioning whether we get enough value out of verifying the signatures to make it worth doing something here, especially since there's no straightforward short-term answer to the supply chain / "root of trust" problems. :disappointed:
I ended up using the Go cosign CLI via its GHCR registry OCI image (with the SHA256 digest, that Dependabot should be able to bump 🤞) for our builds fwiw:
https://github.com/heroku/heroku-buildpack-python/commit/5f8ddb69cae167c5dc4ca8af8d9549ec3f7e80b6
