pip-compile doesn't provide hashes for wheels hosted by simple index servers

[stefansjs]

pip-compile doesn't compute hashes for all index servers, which leads to pip install failure. Specifically, any index server that provides hashes from its json API, will prevent pip-compile for checking if other installation candidates are available from other index servers.

Environment Versions

1. OS Type
2. Python version: $ python -V
3. pip version: $ pip --version
4. pip-tools version: $ pip-compile --version

Steps to replicate

1. host a simple index server
2. add wheels for source packages in pypi, e.g. avro-python3==1.9.2.1
3. list this requirement in requirements.in
4. run pip-compile --generate-hashes -i 'http://<my local index server>.com/' --extra-index-url https://pypi.org/simple
5. run pip install -r requirements.txt

Expected result

The hashes should include the wheel in the local index server.

Actual result

Only hashes from pypi.org are listed in requirements.txt.

Looking at the implementation, it looks like piptools first tries to get hashes from the first index server that implements the json API. The first server that responds with a json blob is taken as the only hash candidates. Files are not manually hashed for any missing files.

In this case, because some files are on pypi, which implements the json API, other index servers are able to provide valid installation candidates to pip, but piptools does not bother to hash them, thus causing a pip installation failure due to the missing hashes.

[stefansjs]

I've got a PR underway. I'm just trying to make sure I add enough test coverage first.

[snmishra]

@stefansjs I'm running into this limitation as well. Any updates on progress? Could I offer assistance?

[stefansjs]

@snmishra I just got back to this PR. It sat on my back burner for a little longer than I'd like to admit. So far my feature is implemented on https://github.com/stefansjs/pip-tools/tree/all_hashes but I don't think it has enough test coverage to ensure that my implementation is right in all cases.

If it's easier, maybe I should start the PR and we can discuss changes in that CR discussion. Do you think that's the best way to discuss?

[snmishra]

@stefansjs Sounds good

[jedie]

Any News here?!?

[atugushev]

Any News here?!?

@jedie The work is in progress, see related PR https://github.com/jazzband/pip-tools/pull/1556.