feat(install): allow conda package subset selection (e.g. `pixi i --skip`) via manifest

[jstriebel]

Problem description

Having the options to (un-)select specific packages introduced in #4404 is great! I'd like to consistently (un-)select specific transitive dependencies which are not needed, without replicating full packages on a custom channel. To do this consistently, I would like to be able to specify them in the pixi.toml manifest. Preferably, the excluded packages should also be removed from the lockfile, or marked specifically there as "not installed".

[lucascolley]

would you be interested in this for conda deps, PyPI deps, or both?

[jstriebel]

I'm only interested in conds deps. For PyPI deps this is less of an issue, as library authors can use extra dependencies for optional dependencies. (I don't think there's a similar concept for conda packages?). This matches the scope of #4404, see description there:

PyPI filtering is unused until we implement ignore for the PyPI installer as well.

My main concern is to have sth. like the --skip/--skip-with-deps install options available in a consistent declarative way.

[lucascolley]

library authors can use extra dependencies for optional dependencies. (I don't think there's a similar concept for conda packages?)

Yeah not really (yet), but you can have multiple outputs from one feedstock, like https://github.com/conda-forge/scipy-stubs-feedstock/blob/main/recipe/recipe.yaml. They just need to have different names.

We do have a proposal for bringing extras to the conda world at https://github.com/conda/ceps/pull/111.

[jstriebel]

We do have a proposal for bringing extras to the conda world at https://github.com/conda/ceps/pull/111.

Very nice! Lookinf forward to it being adapted!

PS: I just checked for uv, the proposal there is to set sys_platform == 'never' for PyPI packages, see https://github.com/astral-sh/uv/issues/9174#issuecomment-2482710850. If a similar method works for pixi I'd find that a usable workaround.

[tdejager]

I was thinking this might also be good as a pixi extension. I mean the first proposal, not the optional dependencies. Don't think we will get to it soon though.

[effigies]

A similar feature recently landed in uv: https://github.com/astral-sh/uv/pull/16528

Previously you could override the dependency with an extra like flask ; python_version < '0'. However, pixi's dependency-overrides doesn't allow qualifiers on version specifiers like that. I would be very interested in this for PyPI as well as conda, as there are a number of packages I use that do not use optional dependencies to gate subsets of behavior.

[tdejager]

Hey @effigies the original comment talked about the installation exclusively. But you are talking about excluding it from the solve as well. Could you give an example of how you would like this to look in the toml?

With uv supporting this this would be technically feasible for PyPI at least and possible for conda.

[effigies]

Looks like uv uses a top-level:

[tool.uv]
exclude-dependencies = ["werkzeug"]

We could do a similar thing:

[tool.pixi]
# Ensure werkzeug doesn't get pulled in by conda or PyPI dependencies
exclude-dependencies = ["werkzeug"]
exclude-pypi-dependencies = ["werkzeug"]

I think it is similar in kind to channel-priority and solve-strategy, so it could also be ~~overridden~~ extended (I think it would be too confusing for a feature to remove exclusions) under features.

---

As to solve vs install, I think that's an interesting question. You could imagine a solve group where one of the environments has a feature that excludes a package and one doesn't, so the package would still need to be taken into account if it is not excluded from all environments in the solve group. That could be useful for having a fully-featured dev environment and a minified production environment where the only difference is the removal of packages known not to be accessed.

[tdejager]

Yeah solve-groups make it more difficult cause in that sense you have to interpret it it as install-only exclusion. While when its not part of a solve-group you can potentially exclude it from the solve altogether basically unlocking that constraint. Which I assume was part of the original reason for that feature in uv?

[effigies]

My understanding is that there were a number of problems people were solving, including broken constraints, renamed packages, and avoiding installing large-but-unused packages. I don't think they considered, or needed to consider, a way to avoid installation that didn't skip resolution as well.

Anyway, I agree that you have two distinct and useful functions:

exclude = ['do-not-resolve']
skip = ['do-not-install']

Exclude may unblock or speed up resolution, while skip removes dependency trees from an environment without modifying the resolution.

I think these can be combined into a single configuration setting (exclude) where the logic turns into:

1. The solve-group takes the intersection of exclusions from the environments in the group and excludes them from resolution entirely.
2. When writing out each environment to the lock file, its exclusion list is treated as skip-with-deps, and the excluded packages are not written to that environment at all.
3. If a resolved dependency is not included in any environment (it is only reached through skipped dependency trees), it can be excluded from the lock file entirely.

That feels ergonomic to me, but I can also see the appeal in keeping them separate for the explicitness. If kept separate, I would expect two environments in the same solve group with distinct exclusions to produce an error.

[tdejager]

Yeah I think I agree, lets write up a small proposal with a manifest specification and how to merge across features into environments. We could consider both things you are saying as both have merit I believe. It would also be good to have some concrete use-cases, I think for conda things are clear as long as optional-dependencies are not a thing. At least in python where there are runtime checks.

If you would have the time to put the proposal in this issue (or a seperate one), I'd be happy to discuss here with the others. Would you be willing to do that? I feel you have a really good grasp of the requirements for this one 🙂

I'd be happy to post it in our discord after for discussion with the community.

[effigies]

If you have a template or previous example of a proposal, I can take a stab at it, but I don't have a huge amount of time to devote to this. This is very much a "nice to have" for me.

[effigies]

Following https://github.com/prefix-dev/pixi/issues/4457 as a template:

---

Problem description

It is common for a package's dependency tree to have transitive dependencies that are never imported. For example, a library that has reporting functionality might require bokeh and matplotlib, but a dependent tool may not use the reporting functionality at all, and have no need for these packages.

A rarer but possible situation is that a transitive dependency is unmaintained but has a maintained replacement (such as PIL and Pillow). The order of installation matters, and if PIL is installed second, it overwrites the newer Pillow.

Finally, it is possible that an undesired dependency makes a resolution impossible, needlessly expensive, or full of odd backtracks that prevent using the latest versions of all of the intended first-level dependencies.

override can help resolve this last case in a limited way, but there is currently no means to remove a dependency entirely from either the resolution or the installation.

Proposal

Pixi should permit two functions:

1. exclude: Remove a package from a solve group. When encountered in a dependency tree, it is ignored, its dependencies are not taken into account, and it is not recorded in a lock file.
2. skip: Prevent a package from being installed into an environment. Its dependencies are also skipped unless they are reachable from some other location in the dependency tree. This is currently possible using the --skip-with-deps installation flag.

exclude implies skip, but not the other way around. The primary use case for skip-without-exclude is a fully-featured development environment that is in the same solve group as a minimal production environment.

Although the two functions are distinct, they may be exposed to developers either through a single configuration option or two.

Two configuration options

In the two configuration option approach, exclude and skip are directly specified in the configuration file:

[tool.pixi]
exclude = ['packageA']

[tool.pixi.feature.production]
skip = ['packageB']

[tool.pixi.environments]
dev = { solve-group = 'default' }
prod = { features = ['production'], solve-group = 'default' }

In this example, packageA is excluded from the "default" solve group, and packageB is removed from the prod environment, post-solve.

Exclusions and skips for an environment are cumulative. The packages excluded/skipped from an environment are the aggregated exclusions/skips from any contributing feature or target.

This approach gives high flexibility, and the solve/install logic is directly reflected in the configuration file. exclude and skip can be defined in any feature or target. If the environments in a solve group have different exclusion sets, the solve group is inconsistent and should raise an error.

One configuration option

In the single configuration option approach, the exclude keyword is used, and the exclusions/skips are derived for each environment.

[tool.pixi]
exclude = ['packageA']

[tool.pixi.feature.production]
exclude = ['packageB']

[tool.pixi.environments]
dev = { solve-group = 'default' }
prod = { features = ['production'], solve-group = 'default' }
fast = { features = ['production'], solve-group = 'fast' }

In this example, packageA is excluded from the "default" solve group, but packageB is not, because it is only excluded in the prod environment, not the dev environment. When installing, dev will contain packageB, but prod will not. Both packageA and packageB are excluded from the "fast" solve group.

The logic of exclude is:

• Exclusions for an environment are cumulative. The packages excluded from an environment include exclusions from any contributing feature or target.
• The exclusions for a solve-group are the intersection of all environments in that solve-group.
• If an environment has additional exclusions beyond its solve-group, those exclusions and their dependency trees are removed post-resolution, and not written to the lock file.

This solves

• Removing large dependencies from environments.
• Removing problematic constraints from solve groups.

Questions

• Do these configuration options need to be duplicated for PyPI and Conda?
• How do people feel about the ergonomics of one vs two options?

Personally, I think the single config option is most ergonomic. It reads to me as excluding from the environment, and a solve group can exclude a package excluded by all environments. However, others may not like that adding/removing an environment to a solve group changes the exclusions.

[ruben-arts]

I really like the proposal!

Do these configuration options need to be duplicated for PyPI and Conda? I think it would be best to do that as we do that for most options. To extend that, I think it makes sense to read the uv configuration here aswell like we do with the sources. To make it easy to switch between the two tools.

How do people feel about the ergonomics of one vs two options? I like the idea of only having exclude more too. The skip makes less sense for me as it only changes how it's installed. I can definitely see a big case for wanting to exclude certain packages, with making sure they're not restricting the solve. E.g. If you don't want it to be installed, you probably really don't want it to mess with the resolution of the packages in the environment.

I find it hard to estimate the value of this work, so please drag interest to this Issue if you find more people!

---

We could already add the skip field like you propose in the two options scenario, as that is fully supported in Pixi already. It would be a quick win, but it might make it harder for future addition to not be a confusing UX and it would extend the complexity upfront.

[tdejager]

I'm wondering about this statement:

If an environment has additional exclusions beyond its solve-group, those exclusions and their dependency trees are removed post-resolution, and not written to the lock file. That means that the solve is still constraint by such dependencies.

From what I can see from uv this is being excluded from the entire solve: https://github.com/astral-sh/uv/blob/59cbc9fe3e10d799499c6abdec12787fcfd4f7a8/crates/uv-resolver/src/resolver/mod.rs#L1871 I think we should also do the same for the conda case. Basically ignore it during the solve, and not remove post-resolution.

[effigies]

@tdejager If you have two environments in a solve group where one excludes a dependency and the other doesn't, you need to do one of:

1. Exclude from solve. One environment in a solve group can eliminate a dependency from all environments in the group.
2. Include in solve, include in install. Environment's explicitly listed exclusions are not respected.
3. Include in solve, exclude from install. Solve group includes all the dependencies needed for all environments, installed packages match each environment's excludes.
4. Error. Inconsistent solve group.

I would prefer 3, but if that's not an option, 4 seems better to me than 1 or 2.