lock command hangs

[ipmb]

Describe the bug

Hi there!

I maintain the Python Package Manager Shootout.

It benchmarks popular Python package managers in a common set of tasks. As of yesterday, the pdm task started timing out after 6 hours on the "lock cold" task: https://github.com/lincolnloop/python-package-manager-shootout/actions/runs/19102887872/job/54579026426

To reproduce

curl -sL requirements.txt https://raw.githubusercontent.com/getsentry/sentry/51281a6abd8ff4a93d2cebc04e1d5fc7aa9c4c11/requirements-base.txt | grep -v -- --index-url > requirements.txt
pdm import -f requirements ../requirements.txt
rm -rf ~/.cache/pip
rm -rf ~/.cache/pdm
rm -rf pdm/.venv pdm/__pypackages__
mkdir -p pdm/__pypackages__
rm -f pdm/pdm.lock
cd pdm; pdm lock

Expected Behavior

It used to finish in ~2 minutes

Environment Information

GitHub Actions with PDM installed via curl -sSL https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py | python3 -

Verbose Command Output

No response

Additional Context

All code is open source and available at https://github.com/lincolnloop/python-package-manager-shootout

Are you willing to submit a PR to fix this bug?

• Yes, I would like to submit a PR.

[frostming]

Sorry, a flawed backtracking algorithm (sarugaku/resolvelib#199) was exposed by recent releases of related packages.

If it's possible, you can add this section to the pdm pyproject.toml:

[tool.pdm.resolution.overrides]
grpcio-status = "<1.72"

[h0nIg]

@frostming if the problem is with resolvelib-1.2.1, can you please release a pdm patch version which pins resolvelib-1.2.0? currently lots of people can not use pdm inside renovate, because of this

[Shegox]

To add on what @h0nIg, the update tool Renovate invokes under the hood: pdm update --no-sync --update-eager and that seems to be causing an endless loop. That the version contstraints don't make sense because aiobotocore[boto3] requires boto3<1.40.62,>=1.40.46 (from aiobotocore@2.25.1) is a problem, but it would be good if pdm notices that at some point and aborts trying to find a resolution. 😅

E.g. with following pyproject.toml:

[project]

requires-python = ">=3.12"

dependencies = [
    "aiobotocore[boto3]==2.25.1",
    "boto3==1.40.68"
]

It ends up in an endless loop:

$ pipx run pdm==2.26.1 update --update-eager -v
... 
pdm.termui: ======== Starting round 260 ========
pdm.termui: Adding new pin: six 1.12.0
pdm.termui: ======== Starting round 261 ========
pdm.termui:   Adding requirement aiobotocore==2.25.1(from aiobotocore 2.25.1)
pdm.termui:   Adding requirement boto3<1.40.62,>=1.40.46(from aiobotocore 2.25.1)
pdm.termui: Candidate rejected: aiobotocore@2.25.1 because it introduces a new requirement boto3<1.40.62,>=1.40.46 that conflicts with other requirements:
    boto3==1.40.68 (from project)
pdm.termui: Conflicts detected:
  boto3<1.40.62,>=1.40.46 (from aiobotocore@2.25.1)
  boto3==1.40.68 (from project)
...
pdm.termui: ======== Starting round 300 ========
pdm.termui: Adding new pin: six 1.12.0
pdm.termui: ======== Starting round 301 ========
pdm.termui:   Adding requirement aiobotocore==2.25.1(from aiobotocore 2.25.1)
pdm.termui:   Adding requirement boto3<1.40.62,>=1.40.46(from aiobotocore 2.25.1)
pdm.termui: Candidate rejected: aiobotocore@2.25.1 because it introduces a new requirement boto3<1.40.62,>=1.40.46 that conflicts with other requirements:
    boto3==1.40.68 (from project)
pdm.termui: Conflicts detected:
  boto3<1.40.62,>=1.40.46 (from aiobotocore@2.25.1)
  boto3==1.40.68 (from project)
...

Here is the output with -vv: pdm-output.log

---

If you just execute it without the --update-eager it fails fast and as expected:

$ pipx run pdm==2.26.1 update

  0:00:01 🔒 Lock failed.  ERROR: Unable to find a resolution for boto3
because of the following conflicts:
  boto3<1.40.62,>=1.40.46 (from aiobotocore@2.25.1)
  boto3==1.40.68 (from project)
To fix this, you could loosen the dependency version constraints in pyproject.toml. See https://pdm-project.org/en/latest/usage/lockfile/#solve-the-locking-failure for more details.
  0:00:01 🔒 Lock failed.
See /Users/xxxx/Library/Logs/pdm/pdm-lock-ocdb_0w7.log for detailed debug log.
[ResolutionError]: Unable to find a resolution
WARNING: Add '-v' to see the detailed traceback

---

Edit: I did let it run and after ~40 min it indeed aborts:

ubuntu@871c87008258:/tmp$ pdm update --update-reuse
  0:00:01 🔒 Lock failed.  ERROR: Unable to find a resolution for boto3
because of the following conflicts:
  boto3<1.40.62,>=1.40.46 (from aiobotocore@2.25.1)
  boto3==1.40.68 (from project)
To fix this, you could loosen the dependency version constraints in pyproject.toml. See https://pdm-project.org/en/latest/usage/lockfile/#solve-the-locking-failure for more details.
  0:00:01 🔒 Lock failed.
See /home/ubuntu/.local/state/pdm/log/pdm-lock-rs9zgn4q.log for detailed debug log.
[ResolutionError]: Unable to find a resolution
WARNING: Add '-v' to see the detailed traceback
ubuntu@871c87008258:/tmp$ pdm update --update-all
  0:00:01 🔒 Lock failed.  ERROR: Unable to find a resolution for boto3
because of the following conflicts:
  boto3<1.40.62,>=1.40.46 (from aiobotocore@2.25.1)
  boto3==1.40.68 (from project)
To fix this, you could loosen the dependency version constraints in pyproject.toml. See https://pdm-project.org/en/latest/usage/lockfile/#solve-the-locking-failure for more details.
  0:00:01 🔒 Lock failed.
See /home/ubuntu/.local/state/pdm/log/pdm-lock-wnjnxjni.log for detailed debug log.
[ResolutionError]: Unable to find a resolution
WARNING: Add '-v' to see the detailed traceback
ubuntu@871c87008258:/tmp$ pdm update --update-eager
  0:41:54 🔒 Lock failed.  The dependency resolution exceeds the maximum loop depth of 10000, there may be some circular dependencies in your project. Try to solve them or increase the `strategy.resolve_max_rounds`
config.
  0:41:54 🔒 Lock failed.
See /home/ubuntu/.local/state/pdm/log/pdm-lock-mk6mc_zb.log for detailed debug log.
[ResolutionError]: The dependency resolution exceeds the maximum loop depth of 10000
WARNING: Add '-v' to see the detailed traceback

[frostming]

@frostming if the problem is with resolvelib-1.2.1, can you please release a pdm patch version which pins resolvelib-1.2.0? currently lots of people can not use pdm inside renovate

I don't think resolvelib 1.2.0 will help, or you can have a try.

[frostming]

Using the same dependency set on pdm 2.26.7 (resolvelib==1.2.1), it fails fast as expected.