Adding optional dependencies when poetry gives a list

[berg2043]

Describe the bug

When you try to add a dependency that defines its dependencies as optional and in a list, it tries to install the optional dependency

To reproduce

pdm add pyiceberg

Expected Behavior

Adds pyiceberg and not ray since ray is optional

Environment Information

PDM version: 2.24.1 Python Interpreter: C:\Users\pberg\Documents\PersonalDev\python\pyiceberg.venv\Scripts\python.exe (3.13) Project Root: C:/Users/pberg/Documents/PersonalDev/python/pyiceberg Local Packages:

{ "implementation_name": "cpython", "implementation_version": "3.13.3", "os_name": "nt", "platform_machine": "AMD64", "platform_release": "11", "platform_system": "Windows", "platform_version": "10.0.22631", "python_full_version": "3.13.3", "platform_python_implementation": "CPython", "python_version": "3.13", "sys_platform": "win32", }

Verbose Command Output

contains private info but here's some highlights

Adding requirement ray<3.0.0,>=2.10.0; python_version >= "3.9"(from pyiceberg 0.9.1)

ray install fails because ray 2.45.0 is not in our private repo it later tries again and fails again

Additional Context

For pyiceberg, it lists ray in its pyproject.toml as such

[tool.poetry.dependencies] ... ray = [ { version = "==2.10.0", python = "<3.9", optional = true }, { version = ">=2.10.0,<3.0.0", python = ">=3.9", optional = true }, ]

all other optional dependencies are ignored by pdm. I would have expected ray to be ignored as well.

Currently, I can work around this with [tool.pdm.resolution] excludes = ["ray"]

Are you willing to submit a PR to fix this bug?

• [ ] Yes, I would like to submit a PR.

[frostming]

This report is not valid, as PDM doesn't communicate with poetry metadata directly, first it will try to find a usable wheel for installation, if not found, it will call the build backend to build a core metadata from the sources. If anything unexpected is present in the build result, it should be considered a problem of the build backend.

[berg2043]

I just had time to dig into this. Based on my current understanding of packages and build tools (which is extremely shallow) pyiceberg doesn't have a wheel for python 3.13 in pypi. This causes pdm to download the tar and build the module from that. Pyiceberg uses poetry.core.masonry.api as it's build-backend. pdm doesn't currently support poetry core due to poetry's lack of PEP 621 support. This may be outdated as poetry says they support it as of September 15th 2024, but I'm woefully uneducated in this area.

To summarize, I think the issue is due to the lack of a wheel for 3.13 and due to PDM not supporting pypoetry core. If that sounds correct to you, feel free to close this out, otherwise I can do more digging as I have time.