bug(pypi): unexpected pixi lock behavior with tool.uv.sources path dependencies

[eightysteele]

Checks

• [x] I have checked that this issue has not already been reported.

• [x] I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

https://github.com/eightysteele/repros/tree/main/pixi/source-dependencies

# pixi lock in research keeps changing
~/repros$ cd pixi/source-dependencies/research/
~/repros/pixi/source-dependencies/research$ pixi lock
✔ Updated lock-file
~/repros/pixi/source-dependencies/research$ pixi lock
✔ Updated lock-file
~/repros/pixi/source-dependencies/research$ pixi lock
✔ Updated lock-file

# pixi lock in modules/c keeps changing
~/repros/pixi/source-dependencies/research$ cd ../modules/c
~/repros/pixi/source-dependencies/modules/c$ pixi lock
✔ Updated lock-file
~/repros/pixi/source-dependencies/modules/c$ pixi lock
✔ Updated lock-file
~/repros/pixi/source-dependencies/modules/c$ pixi lock
✔ Updated lock-file

# pixi lock works as expected in modules/{a,b}
~/repros/pixi/source-dependencies/modules/c$ cd ../a
~/repros/pixi/source-dependencies/modules/a$ pixi lock
✔ Lock-file was already up-to-date
~/repros/pixi/source-dependencies/modules/a$ pixi lock
✔ Lock-file was already up-to-date

~/repros/pixi/source-dependencies/modules/a$ cd ../b
~/repros/pixi/source-dependencies/modules/b$ pixi lock
✔ Lock-file was already up-to-date
~/repros/pixi/source-dependencies/modules/b$ pixi lock
✔ Lock-file was already up-to-date

# pixi lock works as expected in modules/b/cpp
~/repros/pixi/source-dependencies/modules/b$ cd cpp/
~/repros/pixi/source-dependencies/modules/b/cpp$ pixi lock
✔ Lock-file was already up-to-date
~/repros/pixi/source-dependencies/modules/b/cpp$ pixi lock
✔ Lock-file was already up-to-date

Issue description

In the following repro calling pixi lock multiple times in research and modules/c keeps updating the lock file:

https://github.com/eightysteele/repros/tree/main/pixi/source-dependencies

More context in the Discord thread with @tdejager:

https://discord.com/channels/1082332781146800168/1415753479489720434

Expected behavior

I expected pixi lock to update the lock file and then be up-to-date on subsequent pixi lock calls.

[eightysteele]

@tdejager @lucascolley potentially related observation: is it expected that requires_dist contains the absolute (vs. relative) path to the frogrocket-lo module?

- pypi: ./
  name: frogrocket-slam
  version: 0.1.1.dev131+62c95a092
  sha256: b201205d40e56122b5e9a49972a57af4f77e01a3d533b93ea518afdce5d0741b
  requires_dist:
  - frogrocket-lo @ file:///home/aaron/lab/modules/lo
  - jax>=0.5.0
  - numpy>=2,<3.0.0
  - pyyaml>=6.0.2,<7
  - warp-lang>=1.7.1,<1.9
  requires_python: '>=3.9,<3.13'
  editable: true

asking because this has been causing lockfile conflicts/updates on the team (since the path changes when pixi lock is run on other machines) and i thought it might be related to the issue we're seeing here.

[tdejager]

Oh man this one totally fell off my radar. Let me take a look next week. Sorry about that

[tdejager]

Might be that if we use sources instead of a directly specified PyPI dependency this causes an absolute path. Might be a good lead indeed

[eightysteele]

No worries @tdejager! I did some additional looking and I think [tool.uv.sources] might be implicated.

[eightysteele]

@tdejager any inspiration? i'm happy to help make a contribution here if you wanted to point me in the right direction. :)

[tdejager]

Yeah you are right in the hunch that we need to solve the absolute path problem first.

I improved the error message a bit and running on your example:

DEBUG pixi_config: Loading config from /etc/pixi/config.toml
DEBUG pixi_config: Loading config from /Users/tdejager/Library/Application Support/pixi/config.toml
DEBUG pixi_config: Loading config from /Users/tdejager/.pixi/config.toml
DEBUG pixi_config: Loaded config from: /Users/tdejager/.pixi/config.toml
DEBUG pixi_config: Loading config from /private/tmp/repros/pixi/source-dependencies/research/.pixi/config.toml
DEBUG pixi_config: Failed to load local config: /private/tmp/repros/pixi/source-dependencies/research/.pixi/config.toml (error: no file was found at /private/tmp/repros/pixi/source-dependencies/research/.pixi/config.toml)
 WARN rattler_networking::authentication_storage::storage: error reading netrc file from /Users/tdejager/.netrc: could not parse .netc file: Illegal format in 4:11 token: region
 WARN rattler_networking::authentication_storage::storage: error reading netrc file from /Users/tdejager/.netrc: could not parse .netc file: Illegal format in 4:11 token: region
 INFO pixi_core::lock_file::outdated: the dependencies of environment 'default' for platform linux-64 are out of date because failed to convert between pep508 and uv types: expected an (.whl or sdist) but found: '/private/tmp/repros/pixi/source-dependencies/modules/b'

Note the last line, so I think we need to make sure that these relative paths work first and then, then fix the other issues that remains with lock-file invalidation. Sorry about this whole thing its all somewhat broken.

With regards to having that as a contribution, this one might be bit hairy because I'm reckoning it requires changes to rattler (the underlying library) as well. If you DM me on Discord I'd be happy to find something that is a bit lower-hanging fruit though!

[tdejager]

Wait, now that I think of it there might be an intial fix for this first! Let me investigate.

[eightysteele]

Thanks @tdejager, that sounds good! Let's see what the investigation turns up and then I'm happy to DM on Discord for any low-hanging fruit that would be helpful.

[tdejager]

So I fixed the initial problem with this PR: https://github.com/prefix-dev/pixi/pull/4710

This now parses the url we get back from the locking process correctly. However, we now run into another problem, because we do not know from the pep508 requirement if this is an editable (and frankly we shouldn't care per se) we now still have a lock invalidation issue.

Running on your repro I now get:

INFO pixi_core::lock_file::outdated: the pypi dependencies of environment 'default' for platform linux-64 are out of date because expected b and c NOT to be editable but in the lock-file they are

The problem being, that we return these as non-editable from the conversion method. And these are cross-referenced with the editables required and this conflicts. We can solve this (I think), by re-using the data in the lock file to figure out if we want that requirement to be editable or not. I think you could try this contribution, feel free to DM for the details.

[eightysteele]

@tdejager nice, #4710 lgtm. I'm game for trying the contribution—I'll DM you to get the details (I'm on Pacific Time in California, with a tendency to wake up super early and work, so we might be coordinating over some rather strange hours!)