pixi icon indicating copy to clipboard operation
pixi copied to clipboard

Slow conda resolution

Open aochagavia opened this issue 1 year ago • 4 comments

Checks

  • [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

pixi init pixi-experiment
cd pixi-experiment
time pixi add scikit-learn=1.1.1 pandas=1.4.2 pyarrow=5.0.0 matplotlib=3.5.0

Output:

real    2m59.959s
user    3m7.762s
sys     0m28.510s

Issue description

Solving scikit-learn=1.1.1, pandas=1.4.2, pyarrow=5.0.0, matplotlib=3.5.0 takes minutes, which seems too high.

Expected behavior

I'd expect solving to complete much faster, as it does with other requirement combinations (e.g. at least under a minute)

aochagavia avatar Aug 06 '24 19:08 aochagavia

Thanks for the data point! If you have thoughts on how we can improve this Id love to brainstorm!

baszalmstra avatar Aug 06 '24 19:08 baszalmstra

I have no idea why this might be happening, unfortunately... I'll come back and post here if I suddenly have an insight!

aochagavia avatar Aug 07 '24 10:08 aochagavia

Just as a further datapoint, when I use a platform of osx-64, pixi solves in under 40 seconds on a pretty old laptop. However, when I set the platform to linux-64, it grinds away for over 4 minutes. When I run the add command as:

time pixi -vvv  add scikit-learn=1.1.1 pandas=1.4.2 pyarrow=5.0.0 matplotlib=3.5.0

I can see that resolvo keeps on cycling back to lines like:

INFO resolvo::solver: ╤══ Install pyarrow=5.0.0=py310h225c066_35_cpu at level 50 (required by <root>)

over and over again.

synapticarbors avatar Aug 08 '24 04:08 synapticarbors

I was curious:

❯ time pixi add scikit-learn=1.1.1 pandas=1.4.2 pyarrow=5.0.0 matplotlib=3.5.0
✔ Added scikit-learn=1.1.1
✔ Added pandas=1.4.2
✔ Added pyarrow=5.0.0
✔ Added matplotlib=3.5.0

real    3m28.321s
user    3m56.367s
sys     0m47.349s

It's a bit of an odd environment - pinning an ancient version of pyarrow with more modern deps.

dhirschfeld avatar Aug 08 '24 23:08 dhirschfeld

Doing this with the latest pixi (0.29.0) this only takes 9 seconds for me. Closing as magically solved

ruben-arts avatar Sep 18 '24 14:09 ruben-arts