Force locking a specific version without adjusting constraints in pyproject.toml

[tomzx]

• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] I have searched the documentation and believe that my question is not covered.

Feature Request

I have pyproject.toml and poetry.lock files and I'd like to be able to update the poetry.lock file so it uses a specific version of package without having to set or update an explicit constraint in the pyproject.toml file.

For example, I have the following in my pyproject.toml:

[tool.poetry.dependencies]
torch = "^1.3"

I want to update my poetry.lock file to use torch 1.5.0 but still use the ^1.3 constraint. While I can certainly run poetry update --lock torch, this will install the latest constraint compatible version of torch which would be 1.6.0.

The way I think about this is that I want to be able to set an additional constraint that would further limit which torch (and associated packages) would be considered during the poetry update --lock torch call.

Context

I'm building a docker image where I already have torch pinned at a certain version. When I install the poetry dependencies, the poetry.lock file needs to have a corresponding version as to not download torch again at a different version.

If I can run a command such as poetry update --lock torch==1.6.0 prior to poetry install, then I can easily make torch version specific images.

I also want to specify torch in the pyproject.toml file because it is a dependency of the project itself, which can be used as a package in other projects. Stating that torch is a dependency in some documentation instead of an explicit dependency isn't a great solution.

[abn]

@tomzx you might want to just specify your dependency as torch = ">=1.3.0 <1.6.0". However, what you are trying to do sounds like an anti-pattern.

In your case, you might be better off installing the wheel instead of using poetry to install your final project. The whene will have Requires-dist: torch (>=1.3.0 <2.0.0) since you have specified ^1.3. This means that when the wheel is installed, pip will skip updating torch.

[tomzx]

@tomzx you might want to just specify your dependency as torch = ">=1.3.0 <1.6.0".

I definitely could, but I don't want to change my pyproject.toml file, only the poetry.lock file.

However, what you are trying to do sounds like an anti-pattern.

Anti-pattern in what aspect specifically? I think that my suggestion is reasonable if you frame it as someone that wants to update their poetry.lock file, given they have a lax constraint in their pyproject.toml. An example of this is the project being tested on torch==1.5.0, but allowing version torch=^1.3 if installed as a package in a different project.

In your case, you might be better off installing the wheel instead of using poetry

Installing this library as a wheel would mean that none of the lock file defined versions are respected, is that correct? It would only respect whatever is defined in pyproject.toml. From a version management perspective that's not great because I lose control over which exact package version gets installed. But then again my suggested poetry update --lock torch==1.6.0 would also modify the lock file. The benefit of my suggestion is that I would at least have a poetry.lock file which I could inspect (although I guess I could also run a pip freeze and get similar results).

[takeda]

@abn I don't know if that's @tomzx use case, but I think that feature could be useful for certain projects.

For example saltstack is an application that could be installed as project. Having locked dependencies would be beneficial as it could use what the developer worked with.

Unfortunately the way it is used is not via lock file that was used by the developer. It's not typically installed via pip from PyPI. Most of the time it is released as RPM or DEB package that is installed on the system and relies on system dependencies. And that's where problem is.

A Debian will come with different versions of the dependencies than RedHat or FreeBSD. What would be great is an ability to override add custom restrictions on certain dependencies. So developer could update dev environment to have similar versions as are used on a given system.

For example:

poetry update --versions freebsd.txt

The file could contain entries like:

packageA==1.2.3
packageB==4.5.6
packageC==7.8.9

Now if packageD is used but not in a file, then no restrictions are placed, if packageC is listed in a file, but it is not needed then it's just ignored.

I'm wondering if this could be generalized to that scenario without just forcing a single package.

[jchalupka-pan]

I agree that in most cases it is very strange thing to do, but as mentioned in original post, it might be useful for testing purpose. I am assuming that my library is working with dependency ^1.0, newest version is 1.5, but for some reason I need to test against 1.3 (maybe I want to test against all versions).

1. Of course I can modify the pyproject.toml, run poetry lock --no-update && poetry install --sync before testing, but it would be much more convenient if I could do poetry update [email protected], especially when some automation is in place.
2. I could use poetry add [email protected], but the problem is that I have multi-component project and with poetry update I do not need to track which component depends on which, I can just put a list of all (those not added will be skipped when updating).

I didn't look through poetry code, so I might be mistaken, but it is possible that most of required logic is already implemented. Logic for parsing such syntax would be the same as for poetry add, and the command would "just" add new constraints to those already in place when calculating the version.

[jlmarrugom]

Hi! A workaround that I found was to set a versión in the pyproject.toml, that admits the use of the desired fixed version. Then I manually change the used version the poetry.lock .

It is not a good practice since someone that doesn't know this could get confused and have some problems updating dependencies in the future, but you can document it well or advice in the pyproject.toml with a comment on the manually changed library.

pyproject.toml:

essentia = ">=2.1b5"  # fixed manually to "2.1b6.dev1110" version on poetry.lock

poetry.lock (changed manually based on a previously installed version with pip):

[[package]]
name = "essentia"
version = "2.1b6.dev1110"

In my case this workaround was necessary because when poetry was installing this library in the production environment, an error was raised claiming that the library was't found.

I hope it helps you.