Which copy of the SPEC should changes be proposed against?

[dstansby]

I want to contribute a minor clarification to the spec, but it's not clear what the process is to do that. In particular, which file in this GitHub repo should I propose a change against?

I suggest that a new "Contributing" (or similar) section is added at the top level to https://ngff.openmicroscopy.org/, which explains how to 1) propose minor changes/clarifications 2) how to propose major additions or changes (ie. explain or link to the RFC process).

[joshmoore]

Before @normanrz's 0.5 changes, your changes would have gone into /latest/. https://github.com/ome/ngff/issues/271 captured "well, now we need a new mechanism". If you have suggestions on doing that, feel free. Otherwise, you can keep capturing proposals against /0.5/ as you did in gh-277 etc. while this issue (#276+#271) gets worked out.

[dstansby]

i've put a proposal at #281 - do you have context on why latest was removed (cc @normanrz)? It seems quite logical to me to always have a latest folder that changes are made against, and that is just copied and 'frozen' in a folder named after each version when a release is made.

[normanrz]

We were thinking of migrating to a branch-based scheme of managing versions. Instead of 0.4, 0.5, latest folders, there would be 0.4, 0.5, latest branches. But we didn't have the time to fully build it out because we were focussed on getting 0.5 done. Putting a symlink in for latest -> 0.5 solved the need to keep existing links functioning.

[joshmoore]

See #282 for a first POC

[joshmoore]

With #308 merged, I've prepared https://github.com/ome/ngff/compare/v0.5...joshmoore:ngff:dev?expand=1 as a proposal or at least a example of what it would look like to base new spec changes on one of the extracted branches, here v0.5:

https://github.com/ome/ngff/compare/v0.5...joshmoore:ngff:dev?expand=1

[will-moore]

That compare has "Showing 301 changed files with 3,442 additions and 20,132 deletions." Hopefully that could be cleaned-up a bit?

[joshmoore]

Hmmm.... locally it doesn't look quite as bad:

git diff --stat origin/v0.5 origin/dev
 examples/coordSystems/.config.json                            |    3 +
 examples/coordSystems/arrayCoordSys.json                      |   10 ++
 examples/multiscales_strict/multiscales_example_relative.json |   76 ++++++++++++
 examples/subspace/.config.json                                |    3 +
 examples/subspace/subspaceMultidim.json                       |   44 +++++++
 examples/subspace/subspacePermute.json                        |   26 +++++
 examples/transformations/.config.json                         |    3 +
 examples/transformations/affine2d2d.json                      |   14 +++
 examples/transformations/affine2d3d.json                      |   14 +++
 examples/transformations/bijection.json                       |   15 +++
 examples/transformations/bijection_verbose.json               |    7 ++
 examples/transformations/byDimension1.json                    |   17 +++
 examples/transformations/byDimension2.json                    |   42 +++++++
 examples/transformations/byDimensionInvalid1.json             |   17 +++
 examples/transformations/byDimensionInvalid2.json             |   17 +++
 examples/transformations/byDimensionXarray.json               |   39 +++++++
 examples/transformations/coordinates1d.json                   |   14 +++
 examples/transformations/displacement1d.json                  |   14 +++
 examples/transformations/identity.json                        |    9 ++
 examples/transformations/inverseOf.json                       |   17 +++
 examples/transformations/mapAxis1.json                        |   23 ++++
 examples/transformations/mapAxis2.json                        |   23 ++++
 examples/transformations/rotation.json                        |   14 +++
 examples/transformations/scale.json                           |   14 +++
 examples/transformations/sequence.json                        |   17 +++
 examples/transformations/sequenceSubspace1.json               |   17 +++
 examples/transformations/translation.json                     |   14 +++
 examples/transformations/xarrayLike.json                      |   17 +++
 index.bs                                                      | 1094 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 schemas/axes.schema                                           |   62 ++++++++++
 schemas/coordinateTransformations.schema                      |  296 +++++++++++++++++++++++++++++++++++++++++++++++
 schemas/coordinate_systems.schema                             |   30 +++++
 schemas/coordinate_systems_and_transforms.schema              |   45 ++++++++
 schemas/coordinate_transformation.schema                      |  344 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 schemas/image.schema                                          |  169 ++++++++++++++++++++++++++-
 schemas/strict_axes.schema                                    |   30 +++++
 schemas/strict_coordinate_systems.schema                      |   18 +++
 tests/test_validation.py                                      |    5 +
 transform-details.bs                                          |  106 +++++++++++++++++
 39 files changed, 2723 insertions(+), 16 deletions(-)

I'll try merging v0.5 into dev and see what GitHub thinks.

Edit: https://github.com/ome/ngff/compare/v0.5...joshmoore:ngff:dev?expand=1 is definitely easier to read. Likely not technically necessary, but probably also doesn't hurt.

[dstansby]

Wanted to check what the status was here with the new submodules now? There's now a latest folder that seems to point to 0.5, so that doesn't seem like the right folder/files to propose changes.

[clbarnes]

Keeping latest around seems like a feature, so that people can link to the current finished, released, accepted version of the spec in perpetuity. So how about we have a folder for the "evolving" version and just call it something else? dev, in_progress, wip, next or whatever. This would be the full text of the spec (rather than a branch-as-submodule), and should be the target of PRs for the next version. This makes it much easier to see the actual changes a given PR makes, and clashes between PRs, as well as making it clear that a released version is not the correct target for substantive changes. This spec would use a placeholder where any references are made to the version ({{in_progress}} or whatever). When a new version is released, the process (if we're sticking to the current structure otherwise) is

1. make and check out a version branch v0.6
2. delete everything except in_progress/, and move its contents out into the root
3. replace {{in_progress}} with 0.6
4. commit
5. return to main branch
6. add the v0.6 branch as a submodule
7. tag on main (with some prefixed name like ome-zarr_v0.6 so it doesn't conflict with the branch name)

This means that you can track the changes in the spec between versions by looking at the in_progress/ dir in ome-zarr_v* tags (or the version branches, I guess), while still being able to see both specs at once in the repo, and also keeping the benefits of the branch-submodule structure (which are not clear to me but I trust they exist :) ).

[dstansby]

To more concretely summarise where I think this specific issue is at, and propose a postive way forward:

In https://github.com/ome/ngff/issues/276#issuecomment-2507641144 the advice was to propose a change against the 0.5 folder, which I did do in https://github.com/ome/ngff/pull/277, but then that PR there was some general confusion about propsing a change against the 0.5 folder, which makes sense to me - the 0.5 folder is a released version, so not the version to make changes against.

Perhaps the action here then is to create a new 0.6 or latest folder with the development version? I'm happy to put together a PR to move this forward, but since this is likely to affect any outstanding PRs against the next version and I don't 100% understand how the submodule structure introduced in https://github.com/ome/ngff/pull/308 works, I don't want to do this without someone who knows more about how the submodule structure is intended to work giving feedback on the idea.

[clbarnes]

I note that Josh has just pushed https://github.com/ome/ngff-spec. I imagine the idea is to develop on the main branch there, then branch for every new version; fixes which pertain to previous versions can then be raised against the version branch. The current ome/ngff repo would then just be the supporting cruft and aggregation of previous versions (where changes could be proposed for structural changes etc.). This makes more sense to me, and saves a couple of steps in my proposed workflow above.

[mkitti]

I agree that there should be a latest, dev, or develop branch that new pull requests should go into. Only after pull requests are merged into this branch should appropriate changes be backported into released versions if warranted. This is orthogonal to if there should be a distinct repository or not.

Developing on a main branch based in another repository could be a bit confusing. I would rather change the default branch of https://github.com/ome/ngff-spec to one of the above branch names.

[joshmoore]

Good morning, all. (And @clbarnes, good find :smile:) Yesterday morning @jo-mueller and I started looking into that new repository as the home for the 0.x branches/tags/etc. It's not ready yet but the hope is indeed that with the standard release-creation process on that repo, a new version like https://ngff.openmicroscopy.org/spec/0.6.dev2 could be minted as a readthedocs subproject.

The current pointer of ngff-spec is the version "tagged" for the OME2025 meeting -- 0.6.dev1. The URLs, however, do not resolve because that would require yet another submodule in this repo. Our hope would be to:

• get ngff-spec building on readthedocs
• update the URLs to contain a prefix path like /spec/
• report back to everyone with guidelines on making modificaitions

Currently, my inclination is to not try to migrate the 0.1-0.5 subdirectories at least not as part of the initial move to prioritize, as @dstansby says, the ability of folks to propose changes for the upcoming versions.

[jo-mueller]

Eagle eye @clbarnes! I'm also currently exploring a bit over there to integrate the spec text/examples/schemas into a unified webview to make exploring and cross-referencing content easier over there. (Opinions welcome, see this branch)