poetry icon indicating copy to clipboard operation
poetry copied to clipboard

Only search source for designated platform and python

Open JeffreyWardman opened this issue 1 year ago • 4 comments

The below will search the source for all wheels that match the version rather than restricting to the required platform and python version.

[tool.poetry.dependencies]
python = ">=3.10,<3.11.dev0"
torch = { version="2.2.0.dev20231213", source="torch", platform="linux", python="3.10" }

[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/nightly/cu121"
priority = "supplemental"

E.g. it also downloads: https://download.pytorch.org/whl/nightly/cu121/torch-2.2.0.dev20231213%2Bcu121-cp312-cp312-win_amd64.whl

Note the cp312 and win_amd64.

JeffreyWardman avatar Jan 12 '24 00:01 JeffreyWardman

duplicate #7935

dimbleby avatar Jan 12 '24 08:01 dimbleby

torch is particularly bad for this because they do not provide file hashes at the repository, so poetry feels the need to download all files itself to calculate those hashes.

https://github.com/pytorch/pytorch/issues/76557#issuecomment-1750463659 says this was fixed some time ago: but it does not seem to be the case at their nightly repository. Your fastest route to seeing improvement is likely to follow up with the torch folk to get that to happen

dimbleby avatar Jan 12 '24 08:01 dimbleby

Thanks for linking the preexisting issue. I missed it; my apologies. Surely there's a simple way to add a filter=* argument that will skip attempting to download files that don't contain that regex.

JeffreyWardman avatar Jan 12 '24 09:01 JeffreyWardman

if you have a simple fix, please submit a merge request...

dimbleby avatar Jan 12 '24 09:01 dimbleby