pixi icon indicating copy to clipboard operation
pixi copied to clipboard

fix: better out-of source caching and source code propagation

Open tdejager opened this issue 1 month ago • 4 comments

This adds the SourceCodeLocation struct that holds both the location of the manifest and the potential build-source, which is generally the actual source-code itself.

E.g given this pixi.toml (imagine it being part of a workspace with a pixi.toml):

workspace/proj/pixi.toml

[package]
source = "./bar"

Then we get:

  • manifest_source = workspace/proj
  • build_source = ./bar

Which when called using the source_code() method gives you: workspace/proj/bar, in this specific case.

We can use this structure when to pass around and decide if we need the location to the manifest, or to the source code. Like when caching we need to check the source code to see if we need to do rebuilds. For example, in my plotjugger PR constant rebuilds would occur: https://github.com/facontidavide/PlotJuggler/pull/1201. This PR also fixes that behavior.

AI Disclosure

  • I found and fixed the bug as well as create the SourceCodeLocation, I let codex then propagate that struct throughout the codebase.

tdejager avatar Nov 03 '25 14:11 tdejager

I changed the code significantly. I'm trying to make sure that the pinned build_source_spec is always completely resolved and doesn't depend on the manifest_source anymore. That way, when dealing with a pinned build_source_spec, we can just use it as is without having to convert it in different locations.

@tdejager What is the best way to test this?

baszalmstra avatar Nov 04 '25 10:11 baszalmstra

There was a PR on the suite I made: https://github.com/prefix-dev/pixi-build-testsuite/pull/91

And for a real-world test you can try my PR on PlotJuggler. Can find the link if you want but I'm on mobile.

Note that there are also some unit test in pixi that test parts of this, but does not call any builds.

tdejager avatar Nov 04 '25 11:11 tdejager

I changed the code to always make the source_build PinnedSourceSpec relative to the workspace instead of to some undefined root. This works well, but it looks very strange in the lockfile. If you have a git dependency which has a out-of-source build the entire git url is repeated in the package_build_source field. I think it makes sense to store the path relative to the location only in the lock-file.

baszalmstra avatar Nov 04 '25 13:11 baszalmstra

I looked at the changes :) is now the preferred_source an alternative to the build_source or is it only used for the pinning during a pixi update?

tdejager avatar Nov 04 '25 18:11 tdejager

Nooo! It look like the windows test snapshots contain backslashes!

baszalmstra avatar Nov 21 '25 16:11 baszalmstra

Nooo! It look like the windows test snapshots contain backslashes!

Pffff, it was not converting correctly. Hope I pushed the fix now. This PR Is getting on my nerves :P

tdejager avatar Nov 21 '25 19:11 tdejager