pixi icon indicating copy to clipboard operation
pixi copied to clipboard

`pypi-options` not taken over to environments like `channels` when no-default-feature is enabled.

Open zen-xu opened this issue 9 months ago • 6 comments

Checks

  • [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

[project]
name = "playground"
version = "0.1.0"
description = "Add a short description here"
authors = ["ZhengYu, Xu <[email protected]>"]
channels = ["conda-forge"]
platforms = ["osx-arm64"]

[pypi-options]
index-url = "https://pypi.tuna.tsinghua.edu.cn/simple/"

[dependencies]

[feature.py311.dependencies]
python = "3.11.*"

[feature.py311.pypi-dependencies]
ray = ">2.10.0"

[environments]
default = { features = [] }
py311 = { features = ["py311"], solve-group = "py311", no-default-feature = true }

Lockfile

version: 5
environments:
  default:
    channels:
    - url: https://conda.anaconda.org/conda-forge/
    packages: {}
  py311:
    channels:
    - url: https://conda.anaconda.org/conda-forge/
    indexes:
    - https://pypi.org/simple
    packages:
      osx-arm64:
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-h93a5062_5.conda
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/ca-certificates-2024.2.2-hf0a4a13_0.conda
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/libexpat-2.6.2-hebf3989_0.conda
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/libffi-3.4.2-h3422bc3_5.tar.bz2
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/libsqlite-3.45.3-h091b4b1_0.conda
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/libzlib-1.2.13-h53f4e23_5.conda
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/ncurses-6.5-hb89a1cb_0.conda
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/openssl-3.3.0-h0d3ecfb_0.conda
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/python-3.11.9-h932a869_0_cpython.conda
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/readline-8.2-h92ec313_1.conda
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/tk-8.6.13-h5083fa2_1.conda
      - conda: https://conda.anaconda.org/conda-forge/noarch/tzdata-2024a-h0c530f3_0.conda
      - conda: https://conda.anaconda.org/conda-forge/osx-arm64/xz-5.2.6-h57fd34a_0.tar.bz2
      - pypi: https://files.pythonhosted.org/packages/76/ac/a7305707cb852b7e16ff80eaf5692309bde30e2b1100a1fcacdc8f731d97/aiosignal-1.3.1-py3-none-any.whl
      - pypi: https://files.pythonhosted.org/packages/e0/44/827b2a91a5816512fcaf3cc4ebc465ccd5d598c45cefa6703fcf4a79018f/attrs-23.2.0-py3-none-any.whl
      - pypi: https://files.pythonhosted.org/packages/ba/06/a07f096c664aeb9f01624f858c3add0a4e913d6c96257acb4fce61e7de14/certifi-2024.2.2-py3-none-any.whl
      - pypi: https://files.pythonhosted.org/packages/dd/51/68b61b90b24ca35495956b718f35a9756ef7d3dd4b3c1508056fa98d1a1b/charset_normalizer-3.3.2-cp311-cp311-macosx_11_0_arm64.whl
      - pypi: https://files.pythonhosted.org/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl
      - pypi: https://files.pythonhosted.org/packages/41/24/0b023b6537dfc9bae2c779353998e3e99ac7dfff4222fc6126650e93c3f3/filelock-3.14.0-py3-none-any.whl
      - pypi: https://files.pythonhosted.org/packages/5b/9c/f12b69997d3891ddc0d7895999a00b0c6a67f66f79498c0e30f27876435d/frozenlist-1.4.1-cp311-cp311-macosx_11_0_arm64.whl
      - pypi: https://files.pythonhosted.org/packages/e5/3e/741d8c82801c347547f8a2a06aa57dbb1992be9e948df2ea0eda2c8b79e8/idna-3.7-py3-none-any.whl
      - pypi: https://files.pythonhosted.org/packages/c8/2f/324fab4be6fe37fb7b521546e8a557e6cf08c1c1b3d0b4839a00f589d9ef/jsonschema-4.22.0-py3-none-any.whl

Issue description

I have environments with different Python versions (py311[default], py37), but they all use the same internal PyPI index.

env py37 should set no-default-feature = true because it has different dependencies with the default env py311

Expected behavior

all environment should use the default pypi-options

zen-xu avatar May 14 '24 07:05 zen-xu

So there was a long discussion in including specifics from the default feature, but it was decided not to do this. So I think it makes sense the pypi-options are also not included, as they are part of the default feature.

However, you are right that channels and platforms have special behavior because they fall under the project category. Pypi-options do not. We could consider changing this for pypi-options, let's see how more people feel about this.

For now I suggest putting the options in a separate feature and including these in the environments :)

tdejager avatar May 14 '24 07:05 tdejager

the no-default-feature feature creep 😄

pypi extra indexes do serve a quite a similar purpose to conda channels...

olivier-lacroix avatar May 14 '24 09:05 olivier-lacroix

I think pypi-options should be like channels as a project option rather than environment option

zen-xu avatar May 14 '24 10:05 zen-xu

We could change it in .toml but it would be breaking, not saying that we shouldn't but we should be careful to do so. Otherwise, we can give it similar behavior to channels, platforms. It's just if no-default includes actually a whole lot of defaults that kind of starts defeating the purpose of it :)

tdejager avatar May 14 '24 12:05 tdejager

Pinging @pavelzw @0xbe7a our "masters of multi environment logic" for your input.

I would personally be fine with adding the pypi-options to the no default with the same logic as the channels, where redefinition means overwriting the table completely, but acknowledging @olivier-lacroix command that we're creating feature creep. And @tdejager's comment about "defeating the purpose" is very dangerous.

Was the [pypi-options] table in the "feature.default` space the right location or should it maybe have been added to the project?

ruben-arts avatar May 14 '24 14:05 ruben-arts

@ruben-arts I think if we give it the same logic as channels and such, that adding it to the project table makes a lot of sense, that would give a more expected experience. We could deprecate the current usage, and give a suggestion of how to fix for a while :).

For other options like pre-release, no-binary and index-resolution-stratagies, we could potentially make a new table that something like pypi-solve-options that is defined on the default feature level.

tdejager avatar May 14 '24 15:05 tdejager

Do you mind me getting started with this? It's kind of blocking the other options I want to add :)

tdejager avatar May 16 '24 08:05 tdejager

I'm really looking forward to pixi becoming more and more mature, and I can't wait to replace all my poetry projects with pixi.

zen-xu avatar May 16 '24 10:05 zen-xu

I think if we give it the same logic as channels and such, that adding it to the project table makes a lot of sense, that would give a more expected experience

I agree with this

pavelzw avatar May 16 '24 23:05 pavelzw