RFC: auto-generate environment.yaml from pyproject.toml

[drammock]

in #12890 we synchronized (core) dependencies listed in our README with those in pyproject.toml. I think we should do the same with environment.yaml too. I wrote a quick script to compare them:

import toml
import yaml

pyproj = toml.load("pyproject.toml")
# ignore "full" as it's just "full-noqt" plus PyQt6, and for conda we need PySide
ignore = ("dev", "doc", "test", "test_extra", "full", "full-pyqt6")
deps = set(pyproj["project"]["dependencies"])
for section, _deps in pyproj["project"]["optional-dependencies"].items():
    if section not in ignore:
        deps |= set(_deps)
recursive_deps = set(x for x in deps if x.startswith("mne["))
deps -= recursive_deps

with open("environment.yml") as fid:
    deps_env = set(yaml.safe_load(fid)["dependencies"])

print(sorted(deps - deps_env))
print()
print(sorted(deps_env - deps))

Here's the differences:

pyproject.toml	environment.yaml	Notes
antio>=0.4.0	missing
pyobjc-framework-Cocoa>=5.2.0; platform_system=='Darwin'	missing
sip	missing
snirf	missing
threadpoolctl	missing
neo	python-neo	name mismatch
PySide6!=6.7.0	pyside6	pin mismatch
h5io>=0.2.4	h5io	pin mismatch
lazy_loader>=0.3	lazy_loader	pin mismatch
matplotlib>=3.6	matplotlib	pin mismatch
numpy>=1.23,<3	numpy	pin mismatch
scipy>=1.9	scipy	pin mismatch
vtk	vtk>=9.2	pin mismatch
imageio>=2.6.1	imageio	in the YAML file twice, with and without pin
missing	jupyter_client
missing	nbformat
missing	numexpr
missing	pillow
missing	ipython !=8.7.0	in "doc" section of TOML
missing	nbclient	in "test-extra" section of TOML
missing	psutil	in "doc" section of TOML
missing	seaborn-base	seaborn!=0.11.2 in "doc" section of TOML
missing	mne-base	not pip-installable / could go away?
missing	mamba	not pip-installable / makes sense
missing	openblas	not pip-installable / makes sense
missing	pip	not pip-installable / makes sense
missing	spyder-kernels>=1.10.0	not pip-installable / makes sense

Seems like it would be fairly easy to add the deps that are missing from TOML, then generate environment.yaml dynamically like we do the README (in a CI hook that listens for pyproject.toml changes):

• The list of things that we want in environment.yaml that can't go in the TOML file is pretty short (mne-base, mamba, openblas, pip, spyder-kernels), and could go in a small file in the repo root (.conda-env-extras.yaml or so)
• I think we could nix mne-base --- what we spec in pyproject.toml should encompass everything in mne-base (and if it doesn't, we should fix that!). Not saying we should stop providing mne-base, just that we shouldn't need it to be a dep in our main environment.yaml file.
• I think we could nix mamba nowadays (libmamba solver is default for a while now).

[larsoner]

... and if we want to be really complete, we should also make the conda-forge recipe for mne-base 100% consistent with the minimal reqs in pyproject.toml and mne at least more consistent with mne[full] as well:

https://github.com/conda-forge/mne-feedstock/blob/ac4cd905e3e5ac5e667fd707dfff498a60ade511/recipe/meta.yaml#L27-L112