pixi icon indicating copy to clipboard operation
pixi copied to clipboard

disable conda-pypi mapping for some or all channels

Open gzm55 opened this issue 1 year ago • 7 comments

Problem description

can we disable the mapping feature for some channels or all channels?

gzm55 avatar May 07 '24 13:05 gzm55

We could make a way to disable it. Could you also share the reason why you would want to disable it?

tdejager avatar May 07 '24 13:05 tdejager

packages from pypi and conda may be compiled with different flags, such as tensorflow, can we depends on the pypi one. Second, when enable mapping, the resolving and installing progress of the tensorflow with mapping is very very slow, in the mean while, pip install takes less then one minutes.

gzm55 avatar May 07 '24 14:05 gzm55

Sure, but if you add tensorflow to pypi-dependencies it should that one right? I mean you cannot have the conda package and the pypi package at the same time.

tdejager avatar May 07 '24 19:05 tdejager

Would also be great if you could share your pixi.toml or pyproject.toml here :)

tdejager avatar May 07 '24 19:05 tdejager

Would also be great if you could share your pixi.toml or pyproject.toml here :)

[project]
name = "tf"
version = "0.1.0"
description = "Add a short description here"
channels = ["conda-forge"]
platforms = ["linux-64"]
# conda-pypi-map = { "conda-forge" = "empty.json" } # force disable mapping

[tasks]

[dependencies]
python = "3.8.*"
nomkl = ">=1.0,<2"

[system-requirements]
linux = "2.6.32" # centos 6
libc = "2.12"

[pypi-options]
index-url = "https://corp.com/pypi/simple" # a pypi mirror

[pypi-dependencies]
tensorflow = { version = ">=2.2, <2.3" }
grpcio = { version = "<1.47" }
h5py = { version = "<3.8" }

gzm55 avatar May 07 '24 22:05 gzm55

@gzm55 The manifest you provide should use the mapping for any packages, and should definitely not slow down the install.

I tested locally by comparing pixi install with pixi run pip install tensorflow==2.2. Funnily enough the results are exactly the same, most likely because this is simply bottle-necked by the download time.

Could it be that the fetching of the mapping is just slow for you?

ruben-arts avatar May 08 '24 10:05 ruben-arts

@gzm55 The manifest you provide should use the mapping for any packages, and should definitely not slow down the install.

I tested locally by comparing pixi install with pixi run pip install tensorflow==2.2. Funnily enough the results are exactly the same, most likely because this is simply bottle-necked by the download time.

Could it be that the fetching of the mapping is just slow for you?

thx, i will try later to test a pre-downloaded mapping.json.

but the conda tensorflow do not meet our needs for the compiling flags, i think a switch is still a simple solution. Or at least, when the tensorflow is not already in the conda deps set, we should keep the use request.

gzm55 avatar May 08 '24 10:05 gzm55

@gzm55 The manifest you provide should use the mapping for any packages, and should definitely not slow down the install.

I tested locally by comparing pixi install with pixi run pip install tensorflow==2.2. Funnily enough the results are exactly the same, most likely because this is simply bottle-necked by the download time.

Could it be that the fetching of the mapping is just slow for you?

@ruben-arts after some testing, the bottle neck seems to be the network.

gzm55 avatar Jun 16 '24 15:06 gzm55