poetry-core icon indicating copy to clipboard operation
poetry-core copied to clipboard
verified publisher icon python-poetry

Experimental: Add support for building pinned artifacts

Open abn opened this issue 5 years ago • 2 comments

This change allows poetry projects to be configured via pyproject.toml to optionally enable version pinning and also to include nested dependencies for application wheels.

NOTE: This is only a proposal, intended to serve as a basis for internal discussion.

Requires: #74 #75

There are some obvious deficiencies to this appraoch.

  1. The markers are computed on the fly and can be a bit fragile as we rely on weak relations to determine package/dependency hierarchy. This could be improved if the solver dumps the "net" markers into the lock file instead of just the pockage markers. Although, it can be argued that we can "compose" this information from available information. /shrug
  2. Path/Git/URL depednencies are unverified in this appraoch.
  3. The lack of a "root level" dependency notion and range does make this implementation dependent in the pyproject.toml file. So this means no install from lockfile option. Although, we are a step closer to that now.
  4. The way we handle condtional dependencies works, but I am not sure if it is sufficient. We take the "dumb" appraoch and assume that (1) is correct and that the PEP 517 front-end knows what it is doing.

An additional beneifit here is that we can improve the requirements.txt export using this mechanism.

Relates-to: python-poetry/poetry#2778 python-poetry/poetry#2331

To test this from poetry, you can install this branch into your poetry environment, patch the Locker implementation by inherting from poetry.core.lock.locker.Locker ony keeping the _locked_repository method; and then add the following to your pyproject.toml.

[tool.poetry.build.metadata]
dependencies = { lock = true, nested = true}

Once that is done, you can simply use poetry build.

Example Generated Metadata

The content below was generated from the poetry's lock file.

dependencies = { lock = false, nested = false} 

Requires-Dist: cachecontrol[filecache] (>=0.12.4,<0.13.0)
Requires-Dist: cachy (>=0.3.0,<0.4.0)
Requires-Dist: cleo (>=0.8.1,<0.9.0)
Requires-Dist: clikit (>=0.6.2,<0.7.0)
Requires-Dist: crashtest (>=0.3.0,<0.4.0); python_version >= "3.6" and python_version < "4.0"
Requires-Dist: functools32 (>=3.2.3,<4.0.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: futures (>=3.3.0,<4.0.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: glob2 (>=0.6,<0.7); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: html5lib (>=1.0,<2.0)
Requires-Dist: importlib-metadata (>=1.6.0,<2.0.0); python_version < "3.8"
Requires-Dist: keyring (>=18.0.1,<19.0.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: keyring (>=20.0.1,<21.0.0); python_version >= "3.5" and python_version < "3.6"
Requires-Dist: keyring (>=21.2.0,<22.0.0); python_version >= "3.6" and python_version < "4.0"
Requires-Dist: packaging (>=20.4,<21.0)
Requires-Dist: pathlib2 (>=2.3,<3.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: pexpect (>=4.7.0,<5.0.0)
Requires-Dist: pkginfo (>=1.4,<2.0)
Requires-Dist: poetry-core (>=1.0.0a9,<2.0.0)
Requires-Dist: requests (>=2.18,<3.0)
Requires-Dist: requests-toolbelt (>=0.8.0,<0.9.0)
Requires-Dist: shellingham (>=1.1,<2.0)
Requires-Dist: subprocess32 (>=3.5,<4.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: tomlkit (>=0.7.0,<1.0.0)
Requires-Dist: typing (>=3.6,<4.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: virtualenv (>=20.0.26,<21.0.0)
dependencies = { lock = true, nested = false} 

Requires-Dist: cachecontrol[filecache] (==0.12.6)
Requires-Dist: cachy (==0.3.0)
Requires-Dist: cleo (==0.8.1)
Requires-Dist: clikit (==0.6.2)
Requires-Dist: crashtest (==0.3.1); python_version >= "3.6" and python_version < "4.0"
Requires-Dist: functools32 (==3.2.3-2); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: futures (==3.3.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: glob2 (==0.6); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: html5lib (==1.1)
Requires-Dist: importlib-metadata (==1.7.0); python_version < "3.8"
Requires-Dist: keyring (==18.0.1); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: keyring (==20.0.1); python_version >= "3.5" and python_version < "3.6"
Requires-Dist: keyring (==21.3.0); python_version >= "3.6" and python_version < "4.0"
Requires-Dist: packaging (==20.4)
Requires-Dist: pathlib2 (==2.3.5); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: pexpect (==4.8.0)
Requires-Dist: pkginfo (==1.5.0.1)
Requires-Dist: poetry-core (==1.0.0a9)
Requires-Dist: requests (==2.24.0)
Requires-Dist: requests-toolbelt (==0.8.0)
Requires-Dist: shellingham (==1.3.2)
Requires-Dist: subprocess32 (==3.5.4); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: tomlkit (==0.7.0)
Requires-Dist: typing (==3.7.4.3); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: virtualenv (==20.0.30)
dependencies = { lock = true, nested = true} 

Requires-Dist: SecretStorage (==3.1.2); python_version >= "3.6" and python_version < "4.0" and sys_platform == "linux"
Requires-Dist: appdirs (==1.4.4)
Requires-Dist: cachecontrol[filecache] (==0.12.6)
Requires-Dist: cachy (==0.3.0)
Requires-Dist: certifi (==2020.6.20)
Requires-Dist: cffi (==1.14.2)
Requires-Dist: chardet (==3.0.4)
Requires-Dist: cleo (==0.8.1)
Requires-Dist: clikit (==0.6.2)
Requires-Dist: configparser (==4.0.2); python_version == "2.7"
Requires-Dist: contextlib2 (==0.6.0.post1); python_version < "3"
Requires-Dist: crashtest (==0.3.1); python_version >= "3.6" and python_version < "4.0"
Requires-Dist: cryptography (==3.0)
Requires-Dist: distlib (==0.3.1)
Requires-Dist: entrypoints (==0.3); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: enum34 (==1.1.10); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: filelock (==3.0.12)
Requires-Dist: functools32 (==3.2.3-2); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: futures (==3.3.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: glob2 (==0.6); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: html5lib (==1.1)
Requires-Dist: idna (==2.10)
Requires-Dist: importlib-metadata (==1.7.0); python_version < "3.8"
Requires-Dist: importlib-resources (==3.0.0); python_version < "3.7"
Requires-Dist: ipaddress (==1.0.23); python_version < "3"
Requires-Dist: jeepney (==0.4.3); python_version >= "3.6" and python_version < "4.0" and sys_platform == "linux"
Requires-Dist: keyring (==18.0.1); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: keyring (==20.0.1); python_version >= "3.5" and python_version < "3.6"
Requires-Dist: keyring (==21.3.0); python_version >= "3.6" and python_version < "4.0"
Requires-Dist: lockfile (==0.12.2)
Requires-Dist: msgpack (==1.0.0)
Requires-Dist: packaging (==20.4)
Requires-Dist: pastel (==0.2.0)
Requires-Dist: pathlib2 (==2.3.5); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: pexpect (==4.8.0)
Requires-Dist: pkginfo (==1.5.0.1)
Requires-Dist: poetry-core (==1.0.0a9)
Requires-Dist: ptyprocess (==0.6.0)
Requires-Dist: pycparser (==2.20)
Requires-Dist: pylev (==1.3.0)
Requires-Dist: pyparsing (==2.4.7)
Requires-Dist: pywin32-ctypes (==0.2.0); python_version >= "2.7" and python_version < "2.8" and sys_platform == "win32"
Requires-Dist: requests (==2.24.0)
Requires-Dist: requests-toolbelt (==0.8.0)
Requires-Dist: scandir (==1.10.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: secretstorage (==2.3.1); (sys_platform == "linux2" or sys_platform == "linux") and python_version < "2.8" and python_version >= "2.7"
Requires-Dist: shellingham (==1.3.2)
Requires-Dist: singledispatch (==3.4.0.3); python_version < "3.4"
Requires-Dist: six (==1.15.0)
Requires-Dist: subprocess32 (==3.5.4); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: tomlkit (==0.7.0)
Requires-Dist: typing (==3.7.4.3); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: typing-extensions (==3.7.4.2); python_version >= "3.5" and python_full_version < "3.5.4"
Requires-Dist: urllib3 (==1.25.10)
Requires-Dist: virtualenv (==20.0.30)
Requires-Dist: webencodings (==0.5.1)
Requires-Dist: zipp (==1.2.0); python_version < "3.8"
dependencies = { lock = false, nested = true} 

Requires-Dist: SecretStorage (>=3); python_version >= "3.6" and python_version < "4.0" and sys_platform == "linux"
Requires-Dist: appdirs (>=1.4.3,<2)
Requires-Dist: cachecontrol[filecache] (>=0.12.4,<0.13.0)
Requires-Dist: cachy (>=0.3.0,<0.4.0)
Requires-Dist: certifi (>=2017.4.17)
Requires-Dist: cffi (>=1.8,<1.11.3 || >1.11.3)
Requires-Dist: chardet (>=3.0.2,<4)
Requires-Dist: cleo (>=0.8.1,<0.9.0)
Requires-Dist: clikit (>=0.6.2,<0.7.0)
Requires-Dist: configparser (>=3.5); python_version == "2.7"
Requires-Dist: contextlib2; python_version < "3"
Requires-Dist: crashtest (>=0.3.0,<0.4.0); python_version >= "3.6" and python_version < "4.0"
Requires-Dist: cryptography
Requires-Dist: distlib (>=0.3.1,<1)
Requires-Dist: entrypoints; python_version >= "2.7" and python_version < "2.8"
Requires-Dist: enum34 (>=1.1,<2.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: enum34 (>=1.1.10,<2.0.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: enum34; python_version < "3"
Requires-Dist: filelock (>=3.0.0,<4)
Requires-Dist: functools32 (>=3.2.3,<4.0.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: futures (>=3.3.0,<4.0.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: glob2 (>=0.6,<0.7); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: html5lib (>=1.0,<2.0)
Requires-Dist: idna (>=2.5,<3)
Requires-Dist: importlib-metadata (>=1.6.0,<2.0.0); python_version < "3.8"
Requires-Dist: importlib-resources (>=1.0); python_version < "3.7"
Requires-Dist: ipaddress; python_version < "3"
Requires-Dist: jeepney (>=0.4.2); python_version >= "3.6" and python_version < "4.0" and sys_platform == "linux"
Requires-Dist: keyring (>=18.0.1,<19.0.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: keyring (>=20.0.1,<21.0.0); python_version >= "3.5" and python_version < "3.6"
Requires-Dist: keyring (>=21.2.0,<22.0.0); python_version >= "3.6" and python_version < "4.0"
Requires-Dist: lockfile (>=0.9)
Requires-Dist: msgpack (>=0.5.2)
Requires-Dist: packaging (>=20.4,<21.0)
Requires-Dist: pastel (>=0.2.0,<0.3.0)
Requires-Dist: pathlib2 (>=2.3,<3.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: pexpect (>=4.7.0,<5.0.0)
Requires-Dist: pkginfo (>=1.4,<2.0)
Requires-Dist: poetry-core (>=1.0.0a9,<2.0.0)
Requires-Dist: ptyprocess (>=0.5)
Requires-Dist: pycparser
Requires-Dist: pylev (>=1.3,<2.0)
Requires-Dist: pyparsing (>=2.0.2)
Requires-Dist: pywin32-ctypes (<0.1.0 || >0.1.0,<0.1.1 || >0.1.1); python_version >= "2.7" and python_version < "2.8" and sys_platform == "win32"
Requires-Dist: requests (>=2.1</b>8,<3.0)
Requires-Dist: requests-toolbelt (>=0.8.0,<0.9.0)
Requires-Dist: scandir; python_version >= "2.7" and python_version < "2.8"
Requires-Dist: secretstorage (<3); (sys_platform == "linux2" or sys_platform == "linux") and python_version < "2.8" and python_version >= "2.7"
Requires-Dist: secretstorage; python_version >= "3.5" and python_version < "3.6" and sys_platform == "linux"
Requires-Dist: shellingham (>=1.1,<2.0)
Requires-Dist: singledispatch; python_version < "3.4"
Requires-Dist: six
Requires-Dist: six (>=1.4.1)
Requires-Dist: six (>=1.9)
Requires-Dist: six (>=1.9.0,<2)
Requires-Dist: subprocess32 (>=3.5,<4.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: tomlkit (>=0.7.0,<1.0.0)
Requires-Dist: typing (>=3.6,<4.0); python_version >= "2.7" and python_version < "2.8"
Requires-Dist: typing-extensions (>=3.6,<4.0); python_version >= "3.5" and python_full_version < "3.5.4"
Requires-Dist: urllib3 (>=1.21.1,<1.25.0 || >1.25.0,<1.25.1 || >1.25.1,<1.26)
Requires-Dist: virtualenv (>=20.0.26,<21.0.0)
Requires-Dist: webencodings
Requires-Dist: zipp (>=0.4); python_version < "3.8"
Requires-Dist: zipp (>=0.5); python_version < "3.8"

abn avatar Aug 27 '20 01:08 abn

is this going to be accepted?

is this a fat wheel approach?

franziskakuehn avatar Jun 18 '24 10:06 franziskakuehn

is this going to be accepted?

This is quite outdated, so probably not in the near future. Without taking a closer look, I assume that python-poetry/poetry#9428 is a more recent variant of this, which produces more reliable markers but is only implemented in poetry, not in poetry-core; in other words: not in the build-backend.

is this a fat wheel approach?

No, it is about pinning dependencies instead of allowing version ranges.

radoering avatar Jun 18 '24 14:06 radoering