pdm-project
lock command hangs
Describe the bug
Hi there!
I maintain the Python Package Manager Shootout.
It benchmarks popular Python package managers in a common set of tasks. As of yesterday, the pdm task started timing out after 6 hours on the "lock cold" task: https://github.com/lincolnloop/python-package-manager-shootout/actions/runs/19102887872/job/54579026426
To reproduce
curl -sL requirements.txt https://raw.githubusercontent.com/getsentry/sentry/51281a6abd8ff4a93d2cebc04e1d5fc7aa9c4c11/requirements-base.txt | grep -v -- --index-url > requirements.txt
pdm import -f requirements ../requirements.txt
rm -rf ~/.cache/pip
rm -rf ~/.cache/pdm
rm -rf pdm/.venv pdm/__pypackages__
mkdir -p pdm/__pypackages__
rm -f pdm/pdm.lock
cd pdm; pdm lock
Expected Behavior
It used to finish in ~2 minutes
Environment Information
GitHub Actions with PDM installed via curl -sSL https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py | python3 -
Verbose Command Output
No response
Additional Context
All code is open source and available at https://github.com/lincolnloop/python-package-manager-shootout
Are you willing to submit a PR to fix this bug?
- [ ] Yes, I would like to submit a PR.
Sorry, a flawed backtracking algorithm (https://github.com/sarugaku/resolvelib/issues/199) was exposed by recent releases of related packages.
If it's possible, you can add this section to the pdm pyproject.toml:
[tool.pdm.resolution.overrides]
grpcio-status = "<1.72"
@frostming if the problem is with resolvelib-1.2.1, can you please release a pdm patch version which pins resolvelib-1.2.0? currently lots of people can not use pdm inside renovate, because of this
To add on what @h0nIg, the update tool Renovate invokes under the hood: pdm update --no-sync --update-eager and that seems to be causing an endless loop. That the version contstraints don't make sense because aiobotocore[boto3] requires boto3<1.40.62,>=1.40.46 (from [email protected]) is a problem, but it would be good if pdm notices that at some point and aborts trying to find a resolution. 😅
E.g. with following pyproject.toml:
[project]
requires-python = ">=3.12"
dependencies = [
"aiobotocore[boto3]==2.25.1",
"boto3==1.40.68"
]
It ends up in an endless loop:
$ pipx run pdm==2.26.1 update --update-eager -v
...
pdm.termui: ======== Starting round 260 ========
pdm.termui: Adding new pin: six 1.12.0
pdm.termui: ======== Starting round 261 ========
pdm.termui: Adding requirement aiobotocore==2.25.1(from aiobotocore 2.25.1)
pdm.termui: Adding requirement boto3<1.40.62,>=1.40.46(from aiobotocore 2.25.1)
pdm.termui: Candidate rejected: [email protected] because it introduces a new requirement boto3<1.40.62,>=1.40.46 that conflicts with other requirements:
boto3==1.40.68 (from project)
pdm.termui: Conflicts detected:
boto3<1.40.62,>=1.40.46 (from [email protected])
boto3==1.40.68 (from project)
...
pdm.termui: ======== Starting round 300 ========
pdm.termui: Adding new pin: six 1.12.0
pdm.termui: ======== Starting round 301 ========
pdm.termui: Adding requirement aiobotocore==2.25.1(from aiobotocore 2.25.1)
pdm.termui: Adding requirement boto3<1.40.62,>=1.40.46(from aiobotocore 2.25.1)
pdm.termui: Candidate rejected: [email protected] because it introduces a new requirement boto3<1.40.62,>=1.40.46 that conflicts with other requirements:
boto3==1.40.68 (from project)
pdm.termui: Conflicts detected:
boto3<1.40.62,>=1.40.46 (from [email protected])
boto3==1.40.68 (from project)
...
Here is the output with -vv: pdm-output.log
If you just execute it without the --update-eager it fails fast and as expected:
$ pipx run pdm==2.26.1 update
0:00:01 🔒 Lock failed. ERROR: Unable to find a resolution for boto3
because of the following conflicts:
boto3<1.40.62,>=1.40.46 (from [email protected])
boto3==1.40.68 (from project)
To fix this, you could loosen the dependency version constraints in pyproject.toml. See https://pdm-project.org/en/latest/usage/lockfile/#solve-the-locking-failure for more details.
0:00:01 🔒 Lock failed.
See /Users/xxxx/Library/Logs/pdm/pdm-lock-ocdb_0w7.log for detailed debug log.
[ResolutionError]: Unable to find a resolution
WARNING: Add '-v' to see the detailed traceback
Edit: I did let it run and after ~40 min it indeed aborts:
ubuntu@871c87008258:/tmp$ pdm update --update-reuse
0:00:01 🔒 Lock failed. ERROR: Unable to find a resolution for boto3
because of the following conflicts:
boto3<1.40.62,>=1.40.46 (from [email protected])
boto3==1.40.68 (from project)
To fix this, you could loosen the dependency version constraints in pyproject.toml. See https://pdm-project.org/en/latest/usage/lockfile/#solve-the-locking-failure for more details.
0:00:01 🔒 Lock failed.
See /home/ubuntu/.local/state/pdm/log/pdm-lock-rs9zgn4q.log for detailed debug log.
[ResolutionError]: Unable to find a resolution
WARNING: Add '-v' to see the detailed traceback
ubuntu@871c87008258:/tmp$ pdm update --update-all
0:00:01 🔒 Lock failed. ERROR: Unable to find a resolution for boto3
because of the following conflicts:
boto3<1.40.62,>=1.40.46 (from [email protected])
boto3==1.40.68 (from project)
To fix this, you could loosen the dependency version constraints in pyproject.toml. See https://pdm-project.org/en/latest/usage/lockfile/#solve-the-locking-failure for more details.
0:00:01 🔒 Lock failed.
See /home/ubuntu/.local/state/pdm/log/pdm-lock-wnjnxjni.log for detailed debug log.
[ResolutionError]: Unable to find a resolution
WARNING: Add '-v' to see the detailed traceback
ubuntu@871c87008258:/tmp$ pdm update --update-eager
0:41:54 🔒 Lock failed. The dependency resolution exceeds the maximum loop depth of 10000, there may be some circular dependencies in your project. Try to solve them or increase the `strategy.resolve_max_rounds`
config.
0:41:54 🔒 Lock failed.
See /home/ubuntu/.local/state/pdm/log/pdm-lock-mk6mc_zb.log for detailed debug log.
[ResolutionError]: The dependency resolution exceeds the maximum loop depth of 10000
WARNING: Add '-v' to see the detailed traceback
@frostming if the problem is with resolvelib-1.2.1, can you please release a pdm patch version which pins resolvelib-1.2.0? currently lots of people can not use pdm inside renovate
I don't think resolvelib 1.2.0 will help, or you can have a try.