bioformats icon indicating copy to clipboard operation
bioformats copied to clipboard

Add support for reading compressed NDPI tiles

Open melissalinkert opened this issue 1 year ago • 3 comments

This expands on the "precompressed" features added in #3992.

The main goal was to just add compressed tile reading to the NDPI reader, so that conversion from NDPI to DICOM wouldn't require much if any recompression. This required a variety of changes not limited to NDPI though:

  • update JPEGTurboService so that the repackaged compressed tiles can be retrieved, as well as the tile size and counts
  • move general TIFF tile copying logic out of SVSReader, and into MinimalTiffReader (to prevent duplicate logic in NDPIReader)
    • this should also make it easier to support precompressed tiles in other TIFF-based formats later on
  • update NDPIReader to report optimal sizes based on actual IFD/JPEG stream, instead of constant 1024x1024
  • update DicomWriter to allow different tile sizes in each resolution, since NDPI reports different sizes per resolution
  • update NDPIReader to use the saveCompressedBytes API

The end result is that something like this should now work:

bfconvert -noflat -precompressed -compression JPEG CMU-1.ndpi cmu.dcm

and should complete within a few seconds as all tiles can be copied without any recompression.

I would not expect SVS -> DICOM support to change, but it probably makes sense to double-check just in case the refactoring caused any problems.

I don't expect this to fail tests, but excluding for now so this doesn't distract from 7.3.0. Given the API changes in JPEGTurboService, this probably does require a major version.

cc @dclunie, @fedorov

melissalinkert avatar Apr 12 '24 16:04 melissalinkert

I believe the exceptions in https://github.com/ome/bioformats/pull/4181#pullrequestreview-2130154564 are all fixed by 23780de.

melissalinkert avatar Jun 20 '24 23:06 melissalinkert

Loading the generated DICOM files into OMERO for visual assessment and comparison to the original, all datasets worked as expected with the exception of the DICOM dataset created from the SVS file using -precompressed and -compression JPEG-2000 as the import failed with

In this case the pyramid is converted tile-wise, but the extra images are not, which was a problem since the reader's optimal tile sizes are now sent to the writer any time -precompressed is used. I think 4f2a96e326 is one way to fix, but open to other thoughts.

melissalinkert avatar Jun 26 '24 22:06 melissalinkert

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/file-format-to-store-images-using-ngff-coverter/98320/4

imagesc-bot avatar Jun 27 '24 12:06 imagesc-bot