jupyterhub
Support building images with Pixi
Proposed change
Enable building images with pixi using pixi.toml as a configuration file. This is significantly faster (over 10x for an environment I'm working on) compared to micromamba!
Note: This is different than #1339 ; instead of supporting the pixi lockfile format with conda-lock, I want to use pixi as an alternative to conda (i.e. environment.yml and pixi.toml would be mutually exclusive config files) to speed up builds.
Alternative options
- Writing a fully custom
Dockerfile, but that's missing the value of repo2docker!
Who would use this feature?
I would! ;)
I personally believe that the pixi user experience will win out over conda in the long run because it has a superior user experience (i.e. you don't have to manage your specification by hand with pixi). Having it easily available on JupyterHubs will help facilitate the transition.
How much effort will adding it take?
I dunno... yet! I want to find time to dedicate to helping out with repo2docker to learn more. It feels to me like a week of dedicated work could make significant progress. Naive? 😆
Who can do this work?
Knowledge about pixi -- its model is different than conda in that it manages projects, not environments. Project environments are referenced by their location on the filesystem, not by their global name. (https://github.com/prefix-dev/pixi/issues/188). There's a conflict there with how a JupyterHub image has a "global" environment activated by default. Pixi has some affordances for this behavior (https://pixi.sh/latest/reference/cli/pixi/shell-hook/#usage), though. I am a pixi user for the happy path workflow, haven't messed with using pixi in docker images before, but I feel comfortable figuring it out.
Knowledge about Docker, of course :) I feel pretty competent in this space to get my hands dirty.
I think of repo2docker as incorporating best practice rather than defining it. Admittedly this is quite vague since we don't have a formal process for adding new formats to repo2docker.
I think of repo2docker as incorporating best practice rather than defining it. Admittedly this is quite vague since we don't have a formal process for adding new formats to repo2docker.
Hey, Simon! Thanks for the comment. Though I'm not sure I'm clear on the implications. I'm personally less concerned with best practice and more concerned about my images taking 40+ minutes to build. Could you clarify?
I think this is really the same as #1339, since pixi necessarily controls a completely pinned environment (#1339 is not about conda-lock and pixi compatibility, but the general idea of tools that use locked envs like pixi). In particular, to allow users to specify their environment with pixi, we must address https://github.com/jupyterhub/repo2docker/issues/1339#issuecomment-2304253038 . I still think the way to go is to fully respect the lockfile, and then we need to pick an approach for dealing with the fact that the locked env may not be able to launch a jupyter server. I think the least disruptive approach in most cases is a two-env approach - a complete pixi (or conda-lock or pipfile or whatever) locked env for the kernel, pristinely installed as usual, and then a second env for the server, which repo2docker controls.
The cases this won't serve are repos with server extensions, since those won't be installed in the server env. But we only need the two-env approach when the locked env lacks a server, so maybe that will work out okay.
