pip-compile-multi
pip-compile-multi copied to clipboard
peterdemin
Installing a package editably in one envs given diamond hierarchy forces all cases to be editable
In a monorepo setup it's useful to be able to install base packages as editable (-e) in development, though in production you want to be able to install them fully into the site-packages so that you don't need to cart around the source tree separately from the python environment.
It looks like this mostly works for a basic environment hierarchy, however something isn't quite right when there's a diamond hierarchy involved:
┌──→ left ──↘
root ─┼──→ right ─→ dev
└→ base-dev ↗
In the setup I have, a local package is referenced (non-editable) in root and thus transitively by both left and right. base-dev installs it editable. dev just pulls everything together for convenience.
I realise this is a slightly complicated setup, so I've put a reproduce at https://github.com/PeterJCLaw/pip-compile-multi-diamond-hierarchy-bug-demo.
In the failing case, all references to the common utils package are forced to be editable.
Removing either left.in or right.in from dev.in causes the result I want -- only things which depend on base-dev.in actually get forced to have the local package installed editable.
JFYI: I acknowledge the issue but I don't have bandwidth to fix it. You're welcome to propose a solution and I'll review the PR
That's ok. Do you have any suggestions on why this might be working in the ordinary diamond hierarchy? (i.e: the case with only two, not three, legs)
Figured out why removing left.in or right.in from dev.in works -- because then the autoresolve logic can't find a common "sink" file which references all the other environments.
I think I now also understand why this is happening -- because when we do have a common sink and are using --autoresolve, the auto-resolving happens by setting a compatibilities constraints file for pip-compile which includes all the packages from the dev file, which is the editable version of the package.