use a lockfile for "old" CI job
@tsbinns @nordme @CarinaFo @scott-huberty
Here's the PR with placeholders. I discussed with @larsoner and he prefers that we try uv first; if it doesn't work or is unweildy we can switch to pixi (see item 3 below for thoughts on this). In addition to the in-line comments, here are a few notes:
- (perhaps obviously) we'll need to create the lockfile first and commit it to the repo. It should go at
tools/uv-ci-old.lockor similar - For now the "old" environment will only be used on
linux-64systems, so when creating the lockfile we can skip adding support for other platforms. - I'm a bit uncertain if
uv.lockis actually going to work; AFAICTuv lockonly works in uv's "project mode" where it writes a bunch of stuff totool.uv.*tables inpyproject.toml. Right now we don't have such tables (and I don't think we should add them in this PR), so we may need to do something a bit unorthodoxed withuv(e.g., write our pins torequirements-ci-old.txt, use uv to convert that topylock.tomlformat, and restore the environment from the pylock file). See if you can get it to work withuv, but also feel free to experiment withpixito see if the flow is simpler/more natural that way. - We need to think a bit about how to get
mneinstalled (specifically, the local-to-the-CI clone ofmnethat reflects the PR being built). It may be possible to get a relative path in the lockfile, but if not, we may need omitmnein the lockfile and add/install the local-to-ci clone of MNE after the environment has been activated.
hey all, here's a snippet based on what we worked out yesterday, for a script to parse pyproject.toml and generate a requirements file from it after hardening the pins:
from pathlib import Path
import requests
import tomllib
from packaging.requirements import Requirement
from packaging.specifiers import SpecifierSet
with open("/path/to/mne/python/pyproject.toml", "rb") as fid:
toml = tomllib.load(fid)
deps = toml["project"]["dependencies"] + toml["dependency-groups"]["test"]
reqs = [Requirement(dep) for dep in deps]
pinned_reqs = [req for req in reqs if ">" in str(req.specifier)]
unpinned_reqs = list(set(reqs) - set(pinned_reqs))
# TODO triage handling of > vs >= For now, just assert that we needn't worry
assert all([">=" in str(req) for req in pinned_reqs])
for req in pinned_reqs:
req.specifier = SpecifierSet(str(req.specifier).replace(">=", "=="))
# TODO make sure pinned version isn't yanked
# recombine pinned and not-pinned
all_reqs = sorted(pinned_reqs + unpinned_reqs, key=str)
outfile = Path("/path/to/mne/python/tools/requirements-ci-old.txt")
outfile.write_text("\n".join([str(req) for req in all_reqs]))
After that we can in theory run this to get a pylock file:
uv pip compile tools/requirements-ci-old.txt --output-file tools/pylock-ci-old.toml --python-version 3.10
mkdir ~/Desktop/some_temp_dir
cd ~/Desktop/some_temp_dir
uv venv --python 3.10
uv pip sync /path/to/mne/python/tools/pylock-ci-old.toml --python-version 3.10
Note the TODO in the first script about pinned version being yanked; I get a warning about scipy 1.11.0 when doing the uv pip sync line. We'll need to figure out what to do about that. Feel free to try some things out and leave comments here saying what worked/didn't work (or open PRs into my branch if you want)
hey all, here's a snippet based on what we worked out yesterday, for a script to parse pyproject.toml and generate a requirements file from it after hardening the pins:
from pathlib import Path import requests import tomllib from packaging.requirements import Requirement from packaging.specifiers import SpecifierSet with open("/path/to/mne/python/pyproject.toml", "rb") as fid: toml = tomllib.load(fid) deps = toml["project"]["dependencies"] + toml["dependency-groups"]["test"] reqs = [Requirement(dep) for dep in deps] pinned_reqs = [req for req in reqs if ">" in str(req.specifier)] unpinned_reqs = list(set(reqs) - set(pinned_reqs)) # TODO triage handling of > vs >= For now, just assert that we needn't worry assert all([">=" in str(req) for req in pinned_reqs]) for req in pinned_reqs: req.specifier = SpecifierSet(str(req.specifier).replace(">=", "==")) # TODO make sure pinned version isn't yanked # recombine pinned and not-pinned all_reqs = sorted(pinned_reqs + unpinned_reqs, key=str) outfile = Path("/path/to/mne/python/tools/requirements-ci-old.txt") outfile.write_text("\n".join([str(req) for req in all_reqs]))After that we can in theory run this to get a pylock file:
uv pip compile tools/requirements-ci-old.txt --output-file tools/pylock-ci-old.toml --python-version 3.10 mkdir ~/Desktop/some_temp_dir cd ~/Desktop/some_temp_dir uv venv --python 3.10 uv pip sync /path/to/mne/python/tools/pylock-ci-old.toml --python-version 3.10Note the TODO in the first script about pinned version being yanked; I get a warning about scipy 1.11.0 when doing the
uv pip syncline. We'll need to figure out what to do about that. Feel free to try some things out and leave comments here saying what worked/didn't work (or open PRs into my branch if you want)
regarding the uv pip compile command (uv 0.8.22 (ade2bdbd2 2025-09-23) : pylock-ci-old.toml throws an error:
Expected the output filename to start with pylock. and end with .toml (e.g., pylock.toml, pylock.dev.toml); pylock-ci-old.toml
I renamed the file to pylock.ci-old.toml, that fixed it