OME-TIFF: faster initialization when there are lots of planes
Backported from a private PR. Opening to see if repo tests fail, will exclude tomorrow if they do.
This is meant to speed up setId time for cases such as the output of bfconvert "test&sizeZ=4000.fake" test.ome.tiff.
Unit tests fail without 2ebbc58, which I think actually exposes an issue with bfconvert. bfconvert "test&sizeC=2&series=3.fake" -tilex 256 -tiley 256 -channel 1 test-split_%x_%y_%m.ome.tiff results in in this OME-XML:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Warning: this comment is an OME-XML metadata block, which contains crucial dimensional parameters and other important metadata. Please edit cautiously (if at all), and back up the original data before doing so. For more information, see the OME-TIFF web site: https://docs.openmicroscopy.org/latest/ome-model/ome-tiff/. -->
<OME xmlns="http://www.openmicroscopy.org/Schemas/OME/2016-06" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Creator="OME Bio-Formats 6.10.0-SNAPSHOT" UUID="urn:uuid:aa1ec1c8-4f03-4f22-9634-345a61d7275a" xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2016-06 http://www.openmicroscopy.org/Schemas/OME/2016-06/ome.xsd">
<Image ID="Image:0" Name="test">
<Pixels BigEndian="false" DimensionOrder="XYZCT" ID="Pixels:0" Interleaved="false" SignificantBits="8" SizeC="1" SizeT="1" SizeX="256" SizeY="256" SizeZ="1" Type="uint8">
<Channel ID="Channel:0:0" SamplesPerPixel="1">
<LightPath/>
</Channel>
<Channel ID="Channel:0:1" SamplesPerPixel="1">
<LightPath/>
</Channel>
<TiffData FirstC="0" FirstT="0" FirstZ="0" IFD="0" PlaneCount="1">
<UUID FileName="test-split_0_0_0.ome.tiff">urn:uuid:aa1ec1c8-4f03-4f22-9634-345a61d7275a</UUID>
</TiffData>
</Pixels>
</Image>
<Image ID="Image:1" Name="test 2">
<Pixels BigEndian="false" DimensionOrder="XYZCT" ID="Pixels:1" Interleaved="false" SignificantBits="8" SizeC="1" SizeT="1" SizeX="256" SizeY="256" SizeZ="1" Type="uint8">
<Channel ID="Channel:1:0" SamplesPerPixel="1">
<LightPath/>
</Channel>
<Channel ID="Channel:1:1" SamplesPerPixel="1">
<LightPath/>
</Channel>
</Pixels>
</Image>
<Image ID="Image:2" Name="test 3">
<Pixels BigEndian="false" DimensionOrder="XYZCT" ID="Pixels:2" Interleaved="false" SignificantBits="8" SizeC="1" SizeT="1" SizeX="256" SizeY="256" SizeZ="1" Type="uint8">
<Channel ID="Channel:2:0" SamplesPerPixel="1">
<LightPath/>
</Channel>
<Channel ID="Channel:2:1" SamplesPerPixel="1">
<LightPath/>
</Channel>
<TiffData FirstC="0" FirstT="0" FirstZ="0" IFD="0" PlaneCount="1">
<UUID FileName="test-split_1_0_1.ome.tiff">urn:uuid:25c6d8cb-ed9d-4f80-99cf-cb4d0e343882</UUID>
</TiffData>
</Pixels>
</Image>
</OME>
That's not introduced by the PR, and fixing is outside the scope, but noting here for discussion.
Excluding, as this will require further work. The changes here are known to cause problems with https://downloads.openmicroscopy.org/images/OME-TIFF/2016-06/BBBC/, so please do not merge.
The last two commits here are expected to address known performance problems. Removing exclude label so this can be included in tonight's build.
Some additional sanity tests on top of Sebs testing, this time taking a random selection of the remaining public OME-TIFF datasets to ensure no regressions. In all cases the results produced times that were either better or in line with the originals. Overall the results of the PR look to be positive with no notable regressions in any existing datasets.
| tubhiswt-4D | Leica-2.ome.tiff | retina_large.ome.tiff | hcs.companion.ome | MitoCheck | |
|---|---|---|---|---|---|
| 6.10.1 | 0.326 | 0.136 | 0.683 | 0.223 | 0.75 |
| 6.10.1 | 0.311 | 0.121 | 0.863 | 0.22 | 0.805 |
| 6.10.1 | 0.315 | 0.132 | 0.791 | 0.224 | 1.195 |
| 6.10.1 | 0.307 | 0.118 | 0.666 | 0.23 | 0.993 |
| bioformats#3831 | 0.217 | 0.14 | 0.625 | 0.19 | 0.616 |
| bioformats#3831 | 0.249 | 0.146 | 0.803 | 0.211 | 0.808 |
| bioformats#3831 | 0.262 | 0.131 | 0.754 | 0.216 | 0.684 |
| bioformats#3831 | 0.266 | 0.143 | 0.733 | 0.226 | 0.564 |