ITKElastix icon indicating copy to clipboard operation
ITKElastix copied to clipboard

ome zarr notebook

Open thewtex opened this issue 1 year ago • 11 comments

Add OME-Zarr image registration notebook

A simple example that registers one timepoint in an IDR dataset to another.

We use the IDR dataset as it is a familiar reference sample OME-Zarr dataset in the community. This also demonstrates fetching the data dynamically from the cloud.

thewtex avatar Dec 20 '24 23:12 thewtex

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@mstaring @N-Dekker @stefanklein please take a look :pray:

thewtex avatar Jan 16 '25 16:01 thewtex

Very interesting, Matt! I'm still trying to get it to work here locally but I got an error, on import ngff_zarr as nz, below here. Do you have a clue what I might have done wrong?

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[4], line 4
      2 import itk
      3 from itkwidgets import view
----> 4 import ngff_zarr as nz

File ~\.conda\envs\ITKElxPoints\Lib\site-packages\ngff_zarr\__init__.py:6
      1 # SPDX-FileCopyrightText: 2022-present Matt McCormick <[email protected]>
      2 #
      3 # SPDX-License-Identifier: MIT
      5 from .__about__ import __version__
----> 6 from .cli_input_to_ngff_image import cli_input_to_ngff_image
      7 from .config import config
      8 from .detect_cli_io_backend import ConversionBackend, detect_cli_io_backend

File ~\.conda\envs\ITKElxPoints\Lib\site-packages\ngff_zarr\cli_input_to_ngff_image.py:3
      1 import sys
----> 3 import zarr
      4 from dask.array.image import imread as daimread
      5 from rich import print

File ~\.conda\envs\ITKElxPoints\Lib\site-packages\zarr\__init__.py:2
      1 from zarr._version import version as __version__
----> 2 from zarr.api.synchronous import (
      3     array,
      4     consolidate_metadata,
      5     copy,
      6     copy_all,
      7     copy_store,
      8     create,
      9     create_array,
     10     create_group,
     11     empty,
     12     empty_like,
     13     full,
     14     full_like,
     15     group,
     16     load,
     17     ones,
     18     ones_like,
     19     open,
     20     open_array,
     21     open_consolidated,
     22     open_group,
     23     open_like,
     24     save,
     25     save_array,
     26     save_group,
     27     tree,
     28     zeros,
     29     zeros_like,
     30 )
     31 from zarr.core.array import Array, AsyncArray
     32 from zarr.core.config import config

File ~\.conda\envs\ITKElxPoints\Lib\site-packages\zarr\api\synchronous.py:7
      3 from typing import TYPE_CHECKING, Any, Literal
      5 from typing_extensions import deprecated
----> 7 import zarr.api.asynchronous as async_api
      8 import zarr.core.array
      9 from zarr._compat import _deprecate_positional_args

File ~\.conda\envs\ITKElxPoints\Lib\site-packages\zarr\api\asynchronous.py:12
      9 import numpy.typing as npt
     10 from typing_extensions import deprecated
---> 12 from zarr.core.array import Array, AsyncArray, create_array, get_array_metadata
     13 from zarr.core.array_spec import ArrayConfig, ArrayConfigLike
     14 from zarr.core.buffer import NDArrayLike

File ~\.conda\envs\ITKElxPoints\Lib\site-packages\zarr\core\__init__.py:9
      6 from __future__ import annotations
      8 from zarr.core.buffer import Buffer, NDBuffer  # noqa: F401
----> 9 from zarr.core.codec_pipeline import BatchedCodecPipeline

File ~\.conda\envs\ITKElxPoints\Lib\site-packages\zarr\core\codec_pipeline.py:20
     18 from zarr.core.config import config
     19 from zarr.core.indexing import SelectorTuple, is_scalar, is_total_slice
---> 20 from zarr.core.metadata.v2 import _default_fill_value
     21 from zarr.registry import register_pipeline
     23 if TYPE_CHECKING:

File ~\.conda\envs\ITKElxPoints\Lib\site-packages\zarr\core\metadata\__init__.py:3
      1 from typing import TypeAlias, TypeVar
----> 3 from .v2 import ArrayV2Metadata, ArrayV2MetadataDict
      4 from .v3 import ArrayV3Metadata, ArrayV3MetadataDict
      6 ArrayMetadata: TypeAlias = ArrayV2Metadata | ArrayV3Metadata

File ~\.conda\envs\ITKElxPoints\Lib\site-packages\zarr\core\metadata\v2.py:9
      6 from functools import cached_property
      7 from typing import TYPE_CHECKING, TypedDict, cast
----> 9 import numcodecs.abc
     11 from zarr.abc.metadata import Metadata
     13 if TYPE_CHECKING:

File ~\.conda\envs\ITKElxPoints\Lib\site-packages\numcodecs\__init__.py:116
    112 from numcodecs.json import JSON
    114 register_codec(JSON)
--> 116 from numcodecs import vlen as vlen
    117 from numcodecs.vlen import VLenArray, VLenBytes, VLenUTF8
    119 register_codec(VLenUTF8)

File numcodecs\\vlen.pyx:1, in init numcodecs.vlen()

ImportError: numpy.core.multiarray failed to import (auto-generated because you didn't call 'numpy.import_array()' after cimporting numpy; use '<void>numpy._import_array' to disable if you are certain you don't need it).

N-Dekker avatar Jan 20 '25 16:01 N-Dekker

@N-Dekker thanks for testing!

What versions of ngff_zarr, numpy, numcodecs, zarr do you have?

thewtex avatar Jan 20 '25 18:01 thewtex

What versions of ngff_zarr, numpy, numcodecs, zarr do you have?

I just did a brand new install on a new pc. Now it says:

ImportError                               Traceback (most recent call last)
Cell In[1], line 5
      3 import ngff_zarr as nz
      4 from rich import print
----> 5 from zarr.storage import FSStore, LRUStoreCache
      6 import zarr
      7 import copy

ImportError: cannot import name 'FSStore' from 'zarr.storage' (C:\Users\...\.conda\envs\ITKElx\Lib\site-packages\zarr\storage\__init__.py)

I also printed the versions, as follows:

import sys
print(sys.version)
print("itk", itk.__version__)
import itkwidgets
print("itkwidgets", itkwidgets.__version__)
print("ngff_zarr", nz.__version__)
import numpy as np
print("numpy", np.__version__)
import numcodecs
print("numcodecs", numcodecs.__version__)
import zarr
print("zarr", zarr.__version__)

Which says:

3.11.11 | packaged by Anaconda, Inc. | (main, Dec 11 2024, 16:34:19) [MSC v.1929 64 bit (AMD64)]
itk 5.4.0
itkwidgets 0.32.6
ngff_zarr 0.12.1
numpy 2.2.2
numcodecs 0.15.0
zarr 3.0.1

Having pip-installed the "WindowsWheel311.zip" from the ITKElastix CI, from the last commit at the main, at https://github.com/InsightSoftwareConsortium/ITKElastix/actions/runs/12258642912

Do you have a clue?

N-Dekker avatar Jan 24 '25 16:01 N-Dekker

@N-Dekker thanks for the information!

The issue is zarr 3, which was recently released. I'll have to make updates, but for now you could try with zarr 2.

thewtex avatar Jan 24 '25 20:01 thewtex

Thanks Matt. I created a new environment, using zarr version 2:

3.11.11 | packaged by Anaconda, Inc. | (main, Dec 11 2024, 16:34:19) [MSC v.1929 64 bit (AMD64)]
itk 5.4.0
itkwidgets 0.32.6
ngff_zarr 0.12.1
numpy 1.26.4
numcodecs 0.14.1
zarr 2.13.3

Successfully moved one cell further, but unfortunately, now it says:

---------------------------------------------------------------------------
GroupNotFoundError                        Traceback (most recent call last)
Cell In[2], line 7
      5 # Least recently used cache -- avoid re-fetching timepoints
      6 lru_store = LRUStoreCache(fs_store, max_size=1e9)
----> 7 ome_zarr = nz.from_ngff_zarr(lru_store, version='0.3')
      9 # 3D time series
     10 print(ome_zarr)

File ~\.conda\envs\ITKElx-zarr2\Lib\site-packages\ngff_zarr\from_ngff_zarr.py:65, in from_ngff_zarr(store, validate, version)
     59 if version and zarr_version_major >= 3:
     60     format_kwargs = (
     61         {"zarr_format": 2}
     62         if packaging.version.parse(version) < packaging.version.parse("0.5")
     63         else {"zarr_format": 3}
     64     )
---> 65 root = zarr.open_group(store, mode="r", **format_kwargs)
     66 root_attrs = root.attrs.asdict()
     68 if not version:

File ~\.conda\envs\ITKElx-zarr2\Lib\site-packages\zarr\hierarchy.py:1374, in open_group(store, mode, cache_attrs, synchronizer, path, chunk_store, storage_options, zarr_version, meta_array)
   1372         if contains_array(store, path=path):
   1373             raise ContainsArrayError(path)
-> 1374         raise GroupNotFoundError(path)
   1376 elif mode == 'w':
   1377     init_group(store, overwrite=True, path=path, chunk_store=chunk_store)

GroupNotFoundError: group not found at path ''

N-Dekker avatar Jan 27 '25 13:01 N-Dekker

View / edit / reply to this conversation on ReviewNB

N-Dekker commented on 2025-01-27T13:43:42Z ----------------------------------------------------------------

Line #7.    ome_zarr = nz.from_ngff_zarr(lru_store, version='0.3')

from_ngff_zarr returns a Multiscales object, so I would rather name the variable "multiscales" (instead of "ome_zar"). Just like in https://ngff-zarr.readthedocs.io/en/latest/python.html#read-an-ome-zarr


View / edit / reply to this conversation on ReviewNB

N-Dekker commented on 2025-01-27T13:43:42Z ----------------------------------------------------------------

Line #5.    vmin = ome_zarr.metadata.omero.channels[0].window.start

For my understanding, is the index [0] in channels[0] related to the index [0] in images[0]?


View / edit / reply to this conversation on ReviewNB

N-Dekker commented on 2025-01-27T13:43:43Z ----------------------------------------------------------------

Line #2.    view(fixed_image, vmin=vmin, vmax=vmax)

"vmin=vmin" and "vmax=vmax" look non-informative tautologies to me. I would like it better if it would say:

view(fixed_image, vmin=window.start, vmax=window.end)

Assuming window = multiscales.metadata.omero.channels[0].window


My review-notebook-app comments are only minor, stylish. No show-stopper.

I just wish I could make it run here locally!

N-Dekker avatar Jan 27 '25 13:01 N-Dekker