Proposal to make `platforms` and thus the lockfile optional

[ruben-arts]

Problem description

Pixi is not conda.

Pixi has an extreme focus on reproducibility, but that can have some drawbacks. Conda has all the freedom but makes reproducing environment more complex.

This proposal focuses on solving problems related to this.

• Users don't want to add "big" lockfiles to their repos.
• Users want to use environments even if the maintainer is not testing them.
• Users want to get the most optimized binaries for their machine.

Proposal

The main idea is that platforms should be read as solve_for_platform thus without defining the platform in the platforms field it will not be included in the lockfile, but it will able to be used.

If we say native, we mean: With the virtual packages of your machine.

Assume the following manifest(pixi.toml):

[workspace]
platforms = ["linux-64"]

Add dependency

pixi add python

• On linux-64: Adds python to manifest and lockfile and installs it into an environment
• On win-64: Adds python to manifest and to the lockfile for linux-64 and installs it into a "native/non-locked" environment

Assume we now restart with no environment existing

rm .pixi -rf

Install environment

pixi install

• On linux-64: Installs the environment from the pixi.lock
• On win-64: Installs "native/non-locked" environment with a warning

Install native environment

We'll add a new feature, where you can define that you want to install for your native platform. This allows you to install, no matter the lockfile, the best packages for your machine.

pixi install --native

• On linux-64: Installs "native/non-locked" environment
• On win-64: Installs "native/non-locked" environment

Run in environment

pixi run echo hello

• On linux-64: Reinstalls the locked environment.
• On win-64: Runs in "native/non-locked" environment, if there is a need for an installation -> installs with a warning

Avoiding the reinstallation of the lockfile can still be done with --as-is.

pixi install --native && pixi run --as-is echo hello

• On linux-64: Installs and runs "native/non-locked" environment
• On win-64: Installs and runs "native/non-locked" environment

Avoid warnings

If no platforms are defined, you will get no warnings on installation.

Skip full environments from the lockfile

As we already do currently, you'll get the intersection of platforms in an environment of multiple features. So you can skip the full lockfile by saying you don't want any platforms.

[feature.no-lock]
platforms = []

[environments]
lint = { features = ["no-lock"]}

Minimal lockfile experience

Instead of adding a full lockfile, lots of these problems can solved by adding a exclude-newer field.

This solves:

• The ability to share a pixi workspace without testing for a platform.
• Not having a lockfile at all.
• Getting optimized binaries for your machine
• Not having to solve platforms that don't work with your machine (pypi).

Questions:

• Is the presented UX of the CLI and TOML good enough?
• Does this solve all your problems related to the lockfile?
• Should it be --platform native instead of --native as we should also add pixi install --platform ANY_PLATFORM?

[pavelzw]

If no platforms are defined, you will get no warnings on installation

What does this mean?

Something like

[workspace]
platforms = []

or like

[workspace]
# no platforms key

I would b in favor of still needing to specify platforms = [] as pixi's main focus should still be that it creates lockfiles.

---

I'm not a huge fan of the terminology native. How about --ignore-lockfile or --non-locked and if you are on an unsupported platform, write a warning by default with the note "to skip this warning, add the --ignore-lockfile flag"? Maybe don't print a warning if you explicitly do platforms = []

[ruben-arts]

Thank you @pavelzw, We meant the second example. Making platforms an optional field. If you have platforms defined, we expect you to want a lockfile and thus we'll warn you about not having the platform locked.

---

native would not mean the same as ignoring the lockfile, as it would also work when there is no lockfile. It also means, ignore any specification of system-requirements. So having the word lock in the definition seems to miss describe it to me. Open to more convincing or ideas though!

[jaimergp]

This would be amazing for conda-forge. Right now we need to sed-patch our pixi.toml in CI to have this kind of lock-less behavior for "any given platform"

[pavelzw]

when i think of native, i think of the native in native_and_emulated. Maybe something like --bypass-constraints/--ignore-constraints/--unconstrained, --no-env-checks or --skip-checks/--no-check would fit better?

[baszalmstra]

Would --platform=current make more sense @pavelzw ?

[pavelzw]

Yes that would make more sense to me. Although it's still not perfect as it doesn't say "ignore lockfile and solve from scratch" directly.

[maximlt]

I'd also love this. In our open-source libraries, we have decided not to commit the lock file to the repo (at the time the size of the lock file was quite big, and personally I find that all these dependabot/renovate PRs bring a lot of noise for little benefit). We list the 4 main platforms in pixi.toml. So locally running pixi run ... ends up locking for all 4 platforms, overhead I'd be happy to get rid of.

[matthewfeickert]

Assume the following manifest(pixi.toml):

[workspace]
platforms = ["linux-64"]

Add dependency

pixi add python

• On linux-64: Adds python to manifest and lockfile and installs it into an environment
• On win-64: Adds python to manifest and to the lockfile for linux-64 and installs it into a "native/non-locked" environment

@ruben-arts @baszalmstra Just to make this redundantly explicit to make sure I understand, as linux-64 is the only described platform, then:

• On linux-64 the current Pixi behavior continues of resolving all dependencies and recording them in the lock file, and then installing the lock file into the environment.
• On any other platform not listed, the listed platforms are resolved as they currently are and recorded in the lock file and the Pixi workspace dependencies are attempted to be resolved for the current "native" platform and if they succeed they are installed, but there is no recording of the resolution for the "native" platform.

If that is correct, then does that mean the entire "native" platform needs to be resolved every time a Pixi operation would mutate the workspace?

@baszalmstra you had mentioned (on the prefix.dev Discord) that this proposal will make working with CUDA dependencies better. I think I'm missing the main advantage here. Is it just that if you were say working on osx-arm64 but wanting to solve the environment for a target linux-64 server with NVIDIA GPUs instead of having to do something like

pixi init example && cd example
pixi workspace platform add linux-64 osx-arm64
pixi add --feature cpu pytorch-cpu
pixi workspace environment add --feature cpu cpu
pixi upgrade --feature cpu
pixi workspace system-requirements add --feature gpu cuda 12.9
pixi workspace environment add --feature gpu gpu
pixi add --feature gpu --platform linux-64 pytorch-gpu

to reach

[workspace]
channels = ["conda-forge"]
name = "example"
platforms = ["linux-64", "osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]

[feature.cpu.dependencies]
pytorch-cpu = ">=2.8.0,<3"

[feature.gpu.system-requirements]
cuda = "12.9"

[feature.gpu.target.linux-64.dependencies]
pytorch-gpu = ">=2.8.0,<3"

[environments]
cpu = ["cpu"]
gpu = ["gpu"]

, one would instead do something like

pixi init example && cd example
pixi workspace platform add linux-64
pixi add --feature cpu pytorch-cpu
pixi workspace environment add --feature cpu cpu
pixi upgrade --feature cpu
pixi workspace system-requirements add --feature gpu cuda 12.9
pixi workspace environment add --feature gpu gpu
pixi add --feature gpu pytorch-gpu

and arrive at

[workspace]
channels = ["conda-forge"]
name = "example"
platforms = ["linux-64"]
version = "0.1.0"

[tasks]

[dependencies]

[feature.cpu.dependencies]
pytorch-cpu = ">=2.8.0,<3"

[feature.gpu.system-requirements]
cuda = "12.9"

[feature.gpu.dependencies]
pytorch-gpu = ">=2.8.0,<3"

[environments]
cpu = ["cpu"]
gpu = ["gpu"]

with a fully resolved and locked linux-64 workspace and also with an installed (and not locked) cpu environment on the "native" osx-arm64 machine (as osx-arm64 isn't CUDA compatible and so can never resolve the gpu feature requirements)?

So this would address the issue of not being able to set platform specific system-requirements by allowing you to not have to lock the "native" platform into the lock file and so avoid the problem by only defining the platforms where the system-requirements of the feature are valid? This would also mean that you're at the same state as conda for the "native" platform with no reproducibility support (in exchange for smaller lock files)?

[baszalmstra]

Ah my appolagies @matthewfeickert I meant to refer to this proposal: https://github.com/prefix-dev/pixi/issues/4458