micropipenv
micropipenv copied to clipboard
thoth-station
Integrate Sigstore to verify package provenance
Is your feature request related to a problem? Please describe.
As a user of micropipenv, I would like to be able to verify the provenance of packages I install using the sigstore-python library. This way, installation could provide warnings about packages which signature could not be verified or enable rules to be defined to block such packages from being installed. On the Thoth (thamos) side, this would prove really useful as a complement to the security recommendation type.
Another idea was also to enable publishing signed packages to PyPI or other packages indexes via micropipenv using sigstore (similarly to twine but using a better signature method), but I wonder if this as well as the provenance check idea as described above would be too much outside micropipenv's scope.
Hello. Could you please describe in more detail how this should work? Micropipenv uses pip under the hood and installs packages one by one via pip using its internal functions. If I understand it correctly, micropipenv would have to download a package first, download the signature from an online database, verify it and then give this verified package to pip to install it. Is that correct?
I was inclined to close this issue without a detailed description. What is the plan here?
Hello. Could you please describe in more detail how this should work? Micropipenv uses pip under the hood and installs packages one by one via pip using its internal functions. If I understand it correctly, micropipenv would have to download a package first, download the signature from an online database, verify it and then give this verified package to pip to install it. Is that correct?
Hi @frenzymadness, sorry about the late answer. Yes, this would be something similar to the solution you describe, do you think that would be something we could implement? I have not yet taken a deep enough look at micropipenv internals but I am willing to do so if you think this could be a realistic feature.
I'm afraid that this is kinda complex and out of scope for micropipenv. Micropipenv is a lightweight tool designed to work with files produced by tools like pipenv or poetry. Therefore I think that it might make sense to focus on the tools for uploading (twine), storing (warehouse) and installing (pip, pipenv (vendors pip), and poetry) packages. If you can use twine you already know to upload your signed package with some additional metadata and then you can instruct pip to verify the signature during the installation, it'd be useful for much more people and micropipenv can use it as well then.
I am open to more discussions because the topic is really interesting.
you can instruct pip to verify the signature during the installation
I am not very familiar with the way pip handles the verification of signatures as of now; I simply understood that PGP verification by users was abandoned when PyPI started using Warehouse. Do you know if pip/PyPI documentation explains how to verify package signatures during installation? I was not able to find any relevant result.
An other (more general) question I have is: is the goal of micropipenv to have a total API compatibility with pip?
you can instruct pip to verify the signature during the installation
I am not very familiar with the way pip handles the verification of signatures as of now; I simply understood that PGP verification by users was abandoned when PyPI started using Warehouse. Do you know if pip/PyPI documentation explains how to verify package signatures during installation? I was not able to find any relevant result.
I'm afraid it's not possible now. There is PEP 458: https://peps.python.org/pep-0458/ and a very long discussion on top of it https://discuss.python.org/t/pep-458-secure-pypi-downloads-with-package-signing/2648
I don't know all the details but it seems to be slowly progressing. There was a long pause but then somebody funded the effort so it can continue.
An other (more general) question I have is: is the goal of micropipenv to have a total API compatibility with pip?
I'm not aware of any API compatibility between pip and micropipenv. micropipenv uses pip under the hood. Micropipenv can take Pipfile.lock, poetry.lock or requirements.txt, read all important info about project dependencies from them, and then prepare the list of packages for installation. micropipenv then serves the packages one by one to pip and pip does all the magic around picking the correct versions, downloading the package, and installing the package and all its dependencies. micropipenv is here only to read and process the metadata provided by poetry or pipenv.
If we implement the signing and verification to micropipenv, we'd need to also implement a lot of functionality we currently don't care about because it's handled by pip. On the other side of the problem - if poetry, pipenv or pip itself don't support the signing and verification, how would a user configure it for micropipenv?
Thank you for the documentation and answer :+1: I will take examine this more closely
If we implement the signing and verification to micropipenv, we'd need to also implement a lot of functionality we currently don't care about because it's handled by pip. On the other side of the problem - if poetry, pipenv or pip itself don't support the signing and verification, how would a user configure it for micropipenv?
Ok, then I guess this would indeed be out of scope for micropipenv. I am not sure if this feature would be a good fit here and in this case, feel free to close the issue. Otherwise, I would be happy to help contribute to it.
I think that the best course of action now is to read the PEP and think about the possible integration with other tools like pip, poetry, pipenv and then micropipenv in the next round. Also, if you have some special requirement for Thoth project, now it's the right time to discuss them.
Because there is nothing we can do here now, I'm closing the issue.