warehouse
warehouse copied to clipboard
Store attestations for PEP740
Context
This PR follows #16063 and https://github.com/pypi/warehouse/pull/15952 and build on them. After verifying attestations, this PR introduces their storage and retrieval when needed.
Main changes
At upload time
- After attestations have been processed, they are not discarded but stored in using the
storageservice. - A
Provenancefile is generated and stored with the release file
At download time
- When requesting a release using the
simpleAPI (JSON/HTML), if attestations exist for the release :- a new JSON key
provenancewith the sha256 of the provenance file is set - or, a data attribute
data-provenanceis set with the same value
- a new JSON key
Implementations choices
- Attestations are stored based on their hash in buckets:
attestation_sha256_digest = ...
attestation_path = "/'.join(
attestation_sha256_digest[:2],
attestation_sha256_digest[2:4],
attestation_sha256_digest[4:],
f"{release_path}.attestation",
)
- Provenance files are stored along the release file to comply with PEP740
TODOs
- [ ] Improve and fix the tests
- [ ] documentation on the newly created subpackage
/cc @woodruffw @facutuesca
Let put this one on hold until https://github.com/trailofbits/pypi-attestations/pull/36 is merged.
This is looking really good, nice work @DarkaMaul!
To confirm my understanding of the flow here:
- A package is published using TP and contains one (currently just one) attestation in the
attestationsPOST field;- We parse and verify that attestation against the TP identity that published the package file
- We persist the attestation as its own file and content-address it, referencing it from the new
Attestationmodel- We generate the provenance object from the attestation and store it.
Did I get this right?
Exactly