Specify min versions for all the conda packages we use

[mathbunnyru]

Currently, almost everywhere we don't specify package versions during mamba install. Previous behaviour (I changed it a few years ago), specified every version manually.

But the new version has one major flaw: sometimes a package might get downgraded to a really old version. This happened twice already:

• https://github.com/jupyter/docker-stacks/pull/2167
• https://github.com/jupyter/docker-stacks/issues/2170

And I added a fix for notebook and we have to keep it forever, because when a new jupyterlab is released, we might have the same problem.

My proposal - maybe we should pin minimum versions for all the packages?

Upsides:

• we will have less unexpected downgrades (they might still happen during next mamba install, if we don't explicitly pin versions after installing)
• we still don't need to have PRs every time new package is released, like we did a few years ago (because mamba will probably use newer version, if dependencies are resolved)
• probably, less resolution to do, and better logs when some dependency is not resolved
• a bit more control when some security update is released - someone can send a PR where he increases the minimum version (now we always expect our packages to eventually become up-to-date)

Downsides:

• well, we'll need to choose which min versions to specify
• there is some work to make our tests work better when we specify package versions
• there will be more PRs when we see now (every time someone wants to increase min version of some package)
• we haven't tested this way and there might be some issues as well

There are other strategies, for example pin major version, but I don't like it from security perspective: pinning min version gives a bit more control.

[mathbunnyru]

I'm not an expert in terms of dependency resolution and management

@consideRatio @yuvipanda @manics @minrk @JohanMabille @Hind-M @jjerphan maybe you have some thoughts?

[yuvipanda]

I see this has happened twice. My suggestion is wait for it to happen a 3rd time, and then use information from that to make this choice. For example, both notebook, node and mamba are somewhat 'foundational', while a lot of other packages aren't. And if the next time it happens is also similar (say, for numpy), then I'd be more inclined to say 'just do it on a case by case basis'. But if it's something a bit higher up the stack, then maybe we should consider specifying min versions for everything?

So I'd say 'wait and see'

[jjerphan]

For all Semantic Versioned packages, I would recommend solving environments and pinning the min version (included) as the current minor number and the max version as the next major (excluded) as they usually come with breaking changes.

E.g. if SciPy's is resolved to 1.14.1 (the current version), I would pin:

scipy>=1.14,<2

If some packages are known to be API or ABI unstable or if you what to be quite prudent, one might change the max-pinning for the next minor version, e.g. if SciPy were to be that unstable or if you want prudency:

scipy>=1.14,<1.15