URL based dependencies don't generate a hash

[lucaskjaero]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Mac OS X 10.14.6
• Poetry version: 1.0.3
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/lucaskjaero/76424e7ad977613eab9d65623f502bfc

Issue

When adding a normal python dependency, poetry will generate hashes of the downloaded files to put in the lockfile. These can then be given to pip to create reproducible builds. When adding a url dependency, these hashes are not created, and pip cannot successfully install dependencies.

Steps to reproduce

1. poetry add https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz
2. poetry export -f requirements.txt | /venv/bin/pip install -r /dev/stdin

[novemberkilo]

@lucaskjaero is this related to https://github.com/python-poetry/poetry/issues/1631? In essence I don't think poetry generates hashes. It receives them from pypi but not necessarily from private repositories.

Depending on your situation, if you can get away with a requirements file that only contains version numbers and not hashes, you can try:

poetry export --without-hashes -f requirements.txt | /venv/bin/pip install -r /dev/stdin

[lucaskjaero]

@novemberkilo Yes, I think they might have the same root cause. That workaround definitely helps, thanks! It would be nice to get official support for making hashes though, since they are useful.

[mfrlin]

Is there a workaround for this? Can we calculate hash manually and somehow include it in pyproject.toml file?

[polyatail]

@mfrlin You can use pip-compile --generate-hashes from pip_tools on the requirements.txt output from poetry. IIRC, this will fallback to generating hashes when not provided by the source.

[fredrikaverpil]

I believe this problem is also hit if you are using your own private pypi repos. All my regular dependencies have hashes, but for all packages installed from our private pypi repo, the hashes are missing when performing a poetry export.

[AndreyMZ]

I believe this issue should not have low priority because it is a security concern. People use these hashes to mitigate a certain security risk.

The pip install command requires hashes either for all dependencies or for none of them. If a project has at least one URL dependency, then poetry does not produce the hash for it, and we have to ignore hashes for all dependencies, including ones from PyPi (using poetry export --without-hashes). The risk remains not mitigated.

[dimbleby]

this issue belonged in poetry all along: if poetry were to store hashes for url dependencies then the export plugin would write them out

[neersighted]

I'm not sure -- the original ask is for poetry export, even if the code that drives this mostly lives in Poetry. I could go either way, but given all the discussion was around export I put it here. I wouldn't mind opening an issue in Poetry itself that tracks this more directly and references this issue.

[dimbleby]

if there isn't an issue somewhere complaining that poetry fails to store hashes for url dependencies there probably ought to be; and then this would be entirely secondary to that

[dunkmann00]

This should be resolved with https://github.com/python-poetry/poetry/pull/7121