pixi icon indicating copy to clipboard operation
pixi copied to clipboard

Pixi doesn't apply task environments recursively to child tasks

Open yanovs opened this issue 3 months ago • 4 comments

Checks

  • [x] I have checked that this issue has not already been reported.

  • [x] I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

pyproject.toml:

[project]
dependencies = []
name = "pixi_issues"
requires-python = ">= 3.11"
version = "0.1.0"

[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]

[tool.pixi.workspace]
channels = ["conda-forge"]
platforms = ["osx-arm64"]

[tool.pixi.pypi-dependencies]
pixi_issues = { path = ".", editable = true }

[tool.pixi.feature.py311.dependencies]
python = "~=3.11.0"

[tool.pixi.feature.py312.dependencies]
python = "~=3.12.0"

[tool.pixi.environments]
py311 = { features = ["py311"], solve-group = "py311" }
py312 = { features = ["py312"], solve-group = "py312" }

[tool.pixi.tasks]
task0 = "python --version && echo 'Done with task 0'"
task1 = "python --version && echo 'Done with task 1'"
task2 = [{ task = "task0" }, { task = "task1" }]

task3 = [{ task = "task0", environment = "py311" }, { task = "task1", environment = "py312" }]
task5 = [{ task = "task2", environment = "py311" }, { task = "task2", environment = "py312" }]

Issue description

When a "composite task" has no environments specified, Pixi runs the tasks in the default environment, as expected.

When a "composite task" has environments specified for child tasks that are themselves "atomic tasks", Pixi runs as expected.

But when a "composite task" has environments specified for child tasks that are also "composite tasks" themselves, Pixi runs the child task once in the default environment.

> ~/.pixi/bin/pixi-0.56.0 --version
pixi 0.56.0

> ~/.pixi/bin/pixi-0.56.0 run task2
✨ Pixi task (task0 in default): python --version && echo 'Done with task 0'
Python 3.14.0
Done with task 0
✨ Pixi task (task1 in default): python --version && echo 'Done with task 1'
Python 3.14.0
Done with task 1

> ~/.pixi/bin/pixi-0.56.0 run task3
✨ Pixi task (task0 in py311): python --version && echo 'Done with task 0'
Python 3.11.13
Done with task 0
✨ Pixi task (task1 in py312): python --version && echo 'Done with task 1'
Python 3.12.11
Done with task 1

> ~/.pixi/bin/pixi-0.56.0 run task5
✨ Pixi task (task0 in default): python --version && echo 'Done with task 0'
Python 3.14.0
Done with task 0
✨ Pixi task (task1 in default): python --version && echo 'Done with task 1'
Python 3.14.0
Done with task 1

Expected behavior

When running task5, I would've expected Pixi to run task0 and task1 in py311 environment, followed by task0 and task1 in py312 environment. I.e., I think it should be equivalent to:

task5 = [
    { task = "task0", environment = "py311" },
    { task = "task1", environment = "py311" },
    { task = "task0", environment = "py312" },
    { task = "task1", environment = "py312" },
]

yanovs avatar Oct 08 '25 15:10 yanovs

This is currently by design, we feel it could create an issue when you don't what that to be the default behavior. e.g. select a different environment for the dependency.

Do you have idea's on how to fix that?

ruben-arts avatar Oct 13 '25 07:10 ruben-arts

Please let me know if I've understood your comment correctly.

Approach 1

Could the logic be:

  • If the dependency environment of the closest parent edge is set, use that
  • Else if -e is specified, use that
  • Else, use default

E.g.:

task6 = [
    { task = "task2" },
    { task = "task2", environment = "py311" },
    { task = "task2", environment = "py312" },
]

In this case, pixi run task6 would execute: task0 in default, task1 in default, task0 in py311, task1 in py311, task0 in py312, task1 in py312.

But pixi run -e other-env task6 would execute: task0 in other-env, task1 in other-env, task0 in py311, task1 in py311, task0 in py312, task1 in py312. (I.e., only the first invocation of task2 would use -e.)

But if task2 had set environment fields, those would have higher priority since they're "closer".

From my reading and understanding, this functionality is focused on implementing test matrices across Python versions and dependencies. For that reason, we want the dependency environment to hold priority over -e. This would be just extending that idea to "composite tasks".

Finally, I would add that this would make it equivalent to writing a Just recipe like:

test-matrix:
    pixi run task2
    pixi run -e py311 task2
    pixi run -e py312 task2

Approach 2

Just to brainstorm, a different way of thinking about this would be to think of the task environments as just part of the general task argument logic. (Maybe this is what this comment was referencing?)

To do this, we would have to:

  1. Move environment specification from dependency to task
  2. Set a magic variable like {{ default_environment }} to resolve to either -e or default if not set
  3. Add ability to set args in composite tasks (if this isn't already available?)

Then you could (verbosely) configure the tasks like:

[tool.pixi.tasks.task0]
args = [{ "arg" = "environment", "default" = "{{ default_environment }}" }]
cmd = "python --version && echo 'Done with task 0'"
environment = "{{ environment }}"

[tool.pixi.tasks.task1]
args = [{ "arg" = "environment", "default" = "{{ default_environment }}" }]
cmd = "python --version && echo 'Done with task 1'"
environment = "{{ environment }}"

[tool.pixi.tasks.task2]
args = [{ "arg" = "environment", "default" = "{{ default_environment }}" }]
depends-on = [
    { task = "task0", args = [
        { "environment" = "{{ environment }}" },
    ] },
    { task = "task1", args = [
        { "environment" = "{{ environment }}" },
    ] },
]

[tool.pixi.tasks.task6]
depends-on = [
    { task = "task2" },
    { task = "task2", args = [
        { "environment" = "py311" },
    ] },
    { task = "task2", args = [
        { "environment" = "py312" },
    ] },
]

This is mostly for demo purposes, because if we just make environment field and arg a little more special, we would collapse the verbosity to approach 1 above, I think.

yanovs avatar Oct 13 '25 15:10 yanovs

Thank you for the detailed ideas! I'm not sure whether we'll be able to work on this any time soon. The initial approach gives a breaking change in the execution logic, which is a dangerous thing to play with. But it sounds convincing. If you would be able to implement it we could give it a test run and see if it makes sense to do this kind of change.

ruben-arts avatar Oct 20 '25 06:10 ruben-arts

Thanks for your consideration! Though I think Pixi is a great project and would love to give back, I'm not sure I'll have time to work on this any time soon either (especially since I've switched some of my projects to use Just recently).

yanovs avatar Oct 27 '25 13:10 yanovs

I have a similar need when using C++ in multiple environment. I have three tasks test -> build -> configure that can be run with compilers in different environment clang-21, clang-17... When I run test in some environment, I would like to be able to tell the depends-on to be run in the same env. (Additionally I'd like to set a defaut env for test if none is given instead of prompting).

See my example here https://gist.github.com/AntoinePrv/27b6cc763532d971e8db6f588ef4edca

AntoinePrv avatar Nov 18 '25 16:11 AntoinePrv