Allow Requires-Python metadata to be controlled separately from the solver's range of Pythons

[couling]

poetry update ensures that all versions of python will be compatible and report an error if not. From the FAQ:

Unlike pip, Poetry doesn’t resolve for just the Python in the current environment. Instead it makes sure that a dependency is resolvable within the given Python version range in pyproject.toml.

Though I'm about to raise problems with it, I do believe it's a generally good feature. Just a bit tricky to work with.

Problems (in no particular order)

1. I shouldn't need to declare that my package (wheel) conflicts with python >=3.11 if code contained in the wheel doesn't

   • I shouldn't have to re-release my package every time a dependency changes it's own dependencies! The whole point of ^ and ~ dependencies, is I don't need to take action when they release new versions.
   • Artificially constraining package dependencies places a huge burden on the author to monitor dependency grandchildren for changing constraints.
   • As a general rule, it's bad practice to set your own packaging requirements to match those of dependencys' grandchildren. You should instead let the downstream package manager figure it out.
   • For example if I want scipy >= 1.8.1 and that specifies python <3.11. I should not constrain my package to only work on python <3.11, because at some point there will be a scipy > 1.8.1 that is compatible with python >= 3.11. And my package will be compatible with both.

2. How am I supposed to (CI) test with the latest versions of dependences? - do I just abandon poetry and use pip?

   • I ran into this problem because I discovered poetry was silently using scipy 1.6.1 when 1.8.1 would have worked.
   • Using environment markers places a huge responsibility on the author to track all dependencies and their grandchildren, and write ever more complicated rules.
   • poetry update is refusing to install these where pip (and others) wouldn't. So the decision to silently hold back packages creates a terrible burden for testing. Eg scipy 1.8.1 might have been genuinely incompatible, and I wouldn't find out until my downstream user screamed at me.

3. Following the advice given in the error output will trick developers into adding phantom constraints to their wheels.

   • The real constraints come and go as new versions of dependencies are released
   • The addition of artificial constrains fuels dependency hell

4. Confusion / trip hazard:

   • normally one package dependent on foo <3.11 can be completely dependant on foo <4.0 just as long as foo 3.10 is installed. Adding special behaviour for python is bound to trip people up.
   • The project's python dependency is being "abused" as a strict declaration of compatibility rather than a dependency constraint.
   • Nobody experienced in python would expect python 3.30 to be compatible with python 3.7, yet Poetry's own documentation guides you down declaring python = "^3.7". The documentation suggests you declare your package is totally compatible with python 3.99999.1 from a thousand years in the future, but then barks at you when it's dependencies are not.

Requirements

I think these two things would be better

• Separate the two concepts, allowing them to be independently configured:
  • python dependency declared in the wheel and used by downstream packages.
  • python compatibility used by poetry update
• Try to remove special handling of python in dependencies so that all dependencies there are treated the same.

Suggested designs

I'm not fixed on any particular design though my preference is (1).

1. Add an extra field to pyproject.toml eg: python_compatible_range which can be more constrained than the declared python dependency.
   • If python_compatible_range is not set, default its value to the python dependency. (Backwards compatible)
   • If the python dependency is not set, default it from python_compatible_range (don't force users to specify both)
   • If neither is set, raise an error.
   • If both are set, then the python dependency is used for the wheel metadata, and python_compatible_range is used for poetry update.
2. Allow users to pass a python compatible range into poetry update that overrides the python dependency. This option as largely similar to (1), except that it's not stored in pyproject.toml. So defaults and mandatory fields wouldn't change.
3. Include a toggle switch for the special behaviour described Already rejected ;-)

[Monyer]

I fix it by using mkdocs-include-markdown-plugin = { version = "^4.0", python = "^3.8,<3.12" }

[couling]

@Monyer when you say "fixed"... What does that look like in the wheel metadata. More importantly, after building the poetry project as a wheel, what happens when you try to install such a wheel (via pip etc.) on python >=3.12?