RFC: developer meta-environments

[drammock]

This week I needed to recreate my dev env. As you may know, I develop/contribute to several related packages, so I often want/need editable installs of many things and to have all the optional dependencies installed (so I can run their tests, build their docs, etc). This makes things... hard. For example:

• MNE-NIRS isn't really installable: its dependency sphinx_rtd_theme requires docutils<0.21 and its other (nested) dependency readme-renderer requires docutils>=0.21.2
• MNE-LSL forces installation of PyQt5, which can cause headaches if you choose/prefer to install MNE-Python with PyQt6 or PySide
• MNE-Connectivity pins pytest<8 and pydata_sphinx_theme==0.14.1 (specially bad problem for me as I also develop that theme, but others may also want to do local MNE doc builds against current main of that theme)

Possible solutions (please suggest others!)

1. migrate currently maintained MNE tools into a monorepo with a unified environment spec / lockfile (this is possible with e.g. Rye; see https://rdrn.me/postmodern-python/#monorepo)
2. unify a few key things to reduce pain:
   • use the same Qt binding across mne-python, mne-lsl, and mne-qt-browser
   • use the same sphinx theme (and version) across mne-nirs (rtd), mne-lsl (furo), mne-conectivity (pydata==0.14.1) and mne-python (pydata). Ideally we'd also use it for mne-bids-pipeline but I don't expect to win that argument.
3. Do nothing, and make devs use different envs for different packages they work on (I do this too... but it feels weird to need different envs just for working on different MNE packages...)

thoughts?

[larsoner]

use the same Qt binding across mne-python, mne-lsl, and mne-qt-browser

MNE and mne-qt-browser both use qtpy so MNE-LSL could just transition I think.

MNE-Connectivity pins pytest<8

Yes this one is bad -- MNE-Connectivity should be updated not to pin pytest.

use the same sphinx theme

MNE-NIRS doesn't use rtd theme nowadays actually I think the pyproject.toml is just out of date (and even there it's only in the docs spec?).

[larsoner]

... FWIW generally speaking my process is to use a single dev env for as much stuff as possible, and if there are outliers that don't get along with the general Scientific Python practices, I give those a special isolated env.

[mscheltienne]

mne-lsl depends on qtpy, pyqt5 is installed as part of the doc and test extra-key because I do need a qt binding in the environment to run the doc-build and tests (actually need to be checked as the legacy viewer is neither tested through pytest or documented through sphinx-gallery, thus I might be able to entirely drop the qt binding from the extra-keys).

I also tend to use a single environment per project, and when I need an extra package installed in editable mode, I add it to the workspace and install it in the environment I want. I also tend to nuke my environments quite often because it's simpler and faster (especially with uv) to just start fresh than to bother with dependency conflict or to remember what was installed where and when.

[mscheltienne]

FYI, uv pip install mne[full,test,test_extra,doc] reports:

(.venv) mscheltienne@D-1027100163:~/git$ uv pip install mne[full,test,test_extra,doc]
Resolved 265 packages in 187ms
Installed 260 packages in 65ms

Creating environments has never been so fast ;)