Using Cantaloupe with custom ImageReader

[tony-johnson]

HI, I am using Cantaloupe with a custom IIO ImageReader to read a custom image format. To make things work I had to make some small changes to Cantaloupe itself, to support the new format. I am wondering if I missed some way to configure Cantaloupe to use a new IIO reader + image format without having to change code? If not would this capability be something that could usefully be added to Cantaloupe?

[adolski]

Hi @tony-johnson ,

You should be able to use a custom IIO plugin using the processor.imageio.* keys, but the custom format is more of a challenge because of Format being an enum. I'm open to changing that, though.

[tony-johnson]

My thought was perhaps to add a "generic" (or "custom") format to the enumeration, which any IIO plugin could use.

[adolski]

Sorry for the delay--

The processors that use Image I/O readers (Java2dProcessor and JaiProcessor) actually don't use the Image I/O API directly--instead they use some wrapper classes around Image I/O in e.i.l.c.processor.codec. The main reasons for this are:

1. To provide a friendlier API for some features (for example, IIOMetadata is a pain to use)
2. To work around limitations or bugs in the JDK plugins
3. Creating whole IIO plugins is more work

Unfortunately these wrappings defeat the format-agnosticism of Image I/O.

I experimented with getting another format working and my approach involved 4 steps:

1. Add the Format
2. Add an e.i.l.c.p.codec.GenericImageReader implementing ImageReader
3. Modify the default clause in ImageReaderFactory.newImageReader() to return one of those
4. Modify ImageReaderFactory.supportedFormats() to query Image I/O rather than returning a hard-coded Set

That was enough to be able to read simple untiled mono-resolution images without metadata... but not as clean as just adding a plugin.

In the current design, formats are determined by Sources and are used to select a Processor to use. If a Source were to supply Format.UNKNOWN, then the fallback processor would have to be used, and it would have to read the image header to determine the format. So, all of the processors would have to be modified to accept Format.UNKNOWN and figure out the format themselves when necessary. This is doable but I don't know yet if it's what should be done.

So, this turns out to be a tricky problem and I don't have a solution at this time. I am keeping it in mind, though.

[imoutsatsos]

I'm sure that the 'issue: Add support for XYZ image format' will come up again and again." Recently I've encountered the CZI Zeiss format and it would be great to be able to read these images with Cantaloupe (without extracting the images/metdata out of this custom format). Given that BioFormats supports a wealth of image formats from scientific imagers could Cantaloupe use BioFormats as a generic reader? What would be involved? The following OME proposal also makes for an interesting read.

Best regards Ioannis

[adolski]

Hi @imoutsatsos,

I don't know if Bio-Formats already offers Image I/O reader plugins. That would be ideal. If not, then the solution would probably involve a new Processor utilizing a Bio-Formats reader.

There would also need to be some revisions to format detection and redesigning Format as a class rather than an enum. And probably some other things that would become apparent as you get going.

There definitely aren't enough development resources for any of this, unless someone wants to volunteer.

[kinow]

@adolski I've been lurking in this issue, and planning on commenting some day. I've finished most of the work needed in Apache Commons Imaging to get the 1.0 release out of the door (though it's still in alpha).

We got the first wave of early-adopters, and I've going through issues fixing parsing issues, but my next step would be to try using it in Cantaloupe and see if it works and if so, how well it performs.

I think I've got what I need to start working, probably a layer of integration with Image I/O (will try that in a separate repo), or see if this processor would work for Commons Imaging too.

While working on this, I may have a go at looking at the CZI image format (hi @imoutsatsos !). It is not supported by Commons Imaging yet. If it's not too complicated, I could add support to CZI in Commons Imaging, and at the same time try to prepare the required changes in Cantaloupe.

And once again, thanks for the great work on Cantaloupe. Just saw 4.1.3., hope to get some time to test it soon.

Cheers, Bruno