Normalized package versions can cause persistent updates

[eblume]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Ubuntu 18.10
• Poetry version: 0.12.9
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/eblume/b802be1738c206e64501fe68d34f5f19

Issue

STEPS TO REPRODUCE:

$ mkdir poetry_example
$ poetry init  # fill in default values, do not specify any packages
$ poetry add apsw  # snipping the 'dependencies' status text for brevity, here and below
Using version ^3.9 for apsw

Package operations: 1 install, 0 updates, 0 removals

Writing lock file

 - Installing apsw (3.9.2-r1)
$ poetry update
Package operations: 0 installs, 1 update, 0 removals

 - Updating apsw (3.9.2.post1 -> 3.9.2-r1)
$ poetry update
Package operations: 0 installs, 1 update, 0 removals

 - Updating apsw (3.9.2.post1 -> 3.9.2-r1)

I see two issues here:

1. The apsw project is not following the setuptools guidelines, found here: https://setuptools.readthedocs.io/en/latest/setuptools.html#specifying-your-project-s-version - specifically, they are using "-r1" and not the special code "-rc1". However, I still believe there is an issue with poetry here:

2. Poetry is successfully following the rules outlined above to normalize "-r1" to ".post1" in the currently installed version, but it is NOT normalizing "-r1" to ".post1" when checking for an update. And because "r1" is alphanumerically larger than "post1", it considers this to be an update.

Therefore, I believe the fix here is for Poetry to normalize package versions from the available package versions when comparing to the currently installed version.

Note: This issue exists even if I pin the version to apsw==3.9.2-r1. I'm not sure if it is a bug that pinning to apsw==3.9.2.post1 does not work (no version found) - this is an interesting question.

Please let me know if you'd like me to work on a PR to fix this. I think it should be do-able, I just don't know if there's an existing use case that this breaks.

[eblume]

I just want to reiterate that I'm happy to dive in to this, I just want to make sure the maintainers would accept a PR that implements a fix for the case shown in (2) above. I don't know enough about how poetry is used 'in the wild' to know if it would break existing use cases.

[eblume]

I took a look in to this today, and I'm new to the codebase and could use some pointers. I'm trying to write a failing test case. Should this behavior change belong to the Installer, or to the Repository, or to the Solver? I could see it being any of those three. I'm trying to see if I can write a meaningful test for it for the installer.

[sdispater]

@eblume Thanks for taking the time to look into this. The issue comes from the InstalledRepository class which retrieves the currently installed packages from the pip freeze command which translates 3.9.2-r1 to 3.9.2.post1. Poetry does not make that assumption and just leave it as 3.9.2-r1 and this is is where the issue lies.

I think the best approach is to update the internal semver library to understand the r build specification. See here: https://github.com/sdispater/poetry/blob/master/poetry/semver/version.py#L315

[eblume]

Thanks!

I am concerned with adding the r build specification to poetry's semvar as it doesn't address the underlying issue that you can wind up in an infinite loop of no-op updates any time both things are true:

1. The build suffix is not one of the standards-compliant suffices, and
2. The build suffix is alphanumerically less than what it normalizes to (via pip freeze - thanks for that clarification).

I'll see if I can cook up a patch to add the r build spec tonight but I'd like to hear your thoughts on if it might be a better idea to try and capture the original pre-pip freeze-normalized name in the lock file to prevent this issue.

[eblume]

I just want to check in and say that this issue still remains. The attached PR is now effectively dead (and long since closed), but I still see this issue every day and will try and return to it in the future.

I think the next proper step is to introduce a failing test.

[dimbleby]

Seems fixed today: a dependency apsw = "=3.9.2-r1" sticks and is not updated each time.

Please close.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.