SPEC endorsements

[bmcfee]

I took some time today to read through the Scientific Python Ecosystem Coordination (SPEC) recommendations as listed here: https://scientific-python.org/specs/ . This issue is to track and discuss our support (or lack thereof) for each SPEC.

---

SPEC 0 - Minimum supported dependencies

TLDR: https://scientific-python.org/specs/spec-0000/

• Support for Python versions be dropped 3 years after their initial release.
• Support for core package dependencies be dropped 2 years after their initial release.

Our position: We don't currently achieve this, and I don't think we necessarily aspire to either. Rather, our minimum versions are dictated by required functionality and API in the dependencies.

It would be easy for us to adopt this SPEC, but I'm not sure that it's something we want to do necessarily. I don't think the idea, as written, is entirely compatible with long-term reproducibility, but this is definitely a point for discussion.

SPEC 1 - Lazy loading of submodules

We already do this and I think we can endorse it as is.

SPEC 4 - Nightly wheels

There was a start at this during the scipy sprint in 2022, but it never quite got off the ground due to one upstream dependency or another. I do think it's something we should try to do though.

SPEC 6 - Keys to the castle

This one is difficult to summarize concisely. Read more here: https://scientific-python.org/specs/spec-0006/

Of the points listed in the "principles" section in the above link:

1. We could do better on documenting restricted resources, probably.
2. I think we are in compliance with the developer privileges principle, but it could stand some review.
3. We could definitely increase the bus factor on project asset access.
4. I think we're most of the way there via github action publishing?

This one definitely needs discussion.

SPEC 7 - random number generation

TLDR:

• standardizing the usage and interpretation of an rng keyword for seeding, and
• avoiding the use of global state and legacy bitstream generators.

We definitely have a mix of keywords for RNG access in the current release. This could be standardized through a deprecation cycle at the 1.0 checkpoint.

I believe we're otherwise good on global state / bitstreams in the core library code, and #1925 will resolve it similarly in the unit tests.

SPEC 8

TLDR: Complements SPEC 6 and is more targeted at supply chain security (eg pypa distribution).

Broadly, I think we're pretty good on this front through our github action for publishing, but there may be things we can improve upon.

[bmcfee]

SPEC 1 and 7 are now endorsed in the 1.0 branch as of #1960

[bmcfee]

A couple more follow-ups on this:

SPEC0

If we hit a release for 2025.Q3, our min versions would be as follows:

• numpy 1.26 (currently 1.22.3). We currently have an upper version pin at 1.24 for historical doc builds, but I think this is no longer necessary once we move the build limits to 0.10, 0.11, and 1.0.
• scipy 1.11 (currently 1.8). We have an upper pin currently at 1.13.0, so this should be ok. Maybe no longer necessary though.
• matplotlib 3.8 (currently 3.5.0)
• sklearn 1.3 (currently 1.1)
• python 3.9 (maybe 3.10, depending on exact timing)

In general, I think these are all pretty smooth transitions relative to our current min pins. Probably it's fine to nudge our version requirements in this direction anyway, even if we don't fully commit to endorsing the spec.

SPEC4

I want to punt on this one for now, but we should do it at some point.

SPEC6

I'm not sure there's much to do for this. We do have a few secrets (API tokens etc) that are all managed through github. Package publishing is done through trusted publisher (pypi) or merging to conda-forge, so there are no secrets involved. (This can and should be documented in our contributing guidelines.) That said, it would be a good idea to grant access to at least one other person as maintainer on pypi in case I get hit by a bus or something.

The one exception is our twitter account, which we have kept dormant since the hostile takeover. I suppose we could at some point branch back out into social media (bluesky, mastodon, whatever), but I'm not terribly eager to do so.

SPEC8

I don't think we can endorse this one:

Workflows that publish release artifacts should have run triggers that require intentional actions by the release team (e.g., workflow_dispatch in GitHub Actions) and require multiple release team members to approve the workflow to run (c.f. “Use GitHub Actions environments” section below). This is to safeguard the project from any one maintainer having the ability to commit to the default branch and make a release directly.

We do have run triggers, but we just don't have enough active maintainers to make multiple approval practical.

We could still adopt some of the other recommendations though:

• Pin GitHub Actions release workflows to their full release commit SHAs
• Adopt SLSA through use of GitHub Attestations

are both entirely doable.

Everything else I think we're more or less in compliance with already.