tiled icon indicating copy to clipboard operation
tiled copied to clipboard

Add notion of multichannel images to array structure

Open danielballan opened this issue 3 years ago • 1 comments

This was inspired by a demo by @prjemian.

Proposal

Add a new optional key (channels?) to ArrayStructure indicated the array axis containing (color) channels such as RGB, RBG, RGBA, CMYK, etc. This is also useful in the context of multichannel imaging where the channels aren't "colors" in the traditional sense but different signals.

This could be used to by some exporters---in particular, of course, the image format exporters like PNG---to produce color images. It would give all clients more context for how to interpret the data.

The details of how to spell this remain to be worked out.

Alternatives considered

  • Just leave this implicit. That leaves clients displaying color images as 3-frame stacks of monochrome images. Seems bad.
  • Leave it up to the metadata. Image formats like TIFF and JPEG have different ways of parametrizing and spelling this. Tiled should provide a consistent view on it.
  • Put it in dims. ArrayStructure already carries a dims attribute, an (optional) list of strings mean to label each dimension of the array. Currently, the content of dims has no meaning to the server. If we made some dims like "RGB" have special meaning, we would be entering new territory. For both dims and metadata, I think there is value is reserving these are "user space", whose content will never change the behavior of the server.

danielballan avatar Sep 12 '22 20:09 danielballan

More thoughts on why it's important to expose color channels as part of structure:

  • In a sense the "topology" of a multichannel image array is different than if it's a stack of 3 images.
  • Formats like JPEG apply lossy compression to the structure in a way that is fundamentally tied to the human visual system. It wouldn't make sense to expose just any (3, N, M) array as JPEG, and the server should refuse to do it.

danielballan avatar Sep 12 '22 20:09 danielballan