ome
NDPI: IndexOutOfBoundsException when calling getTile in JPEGTurboServiceImpl
Issue was raised on forum thread https://forum.image.sc/t/hamamatsu-ndpi-files-will-not-open-with-bio-formats/69266/3 A sample file has been provided which reproduces the error using Bio-Formats 6.10.0
The full stack trace is:
Exception in thread "BioFormats-ThumbLoader" java.lang.IndexOutOfBoundsException: Index: 6600, Size: 6480
at java.util.ArrayList.rangeCheck(ArrayList.java:659)
at java.util.ArrayList.get(ArrayList.java:435)
at loci.formats.services.JPEGTurboServiceImpl.getTile(JPEGTurboServiceImpl.java:297)
at loci.formats.services.JPEGTurboServiceImpl.getTile(JPEGTurboServiceImpl.java:251)
at loci.formats.in.NDPIReader.openBytes(NDPIReader.java:217)
at loci.formats.FormatReader.openBytes(FormatReader.java:919)
at loci.formats.FormatReader.openBytes(FormatReader.java:890)
at loci.formats.FormatTools.openThumbBytes(FormatTools.java:1308)
at loci.formats.in.NDPIReader.openThumbBytes(NDPIReader.java:247)
at loci.formats.ReaderWrapper.openThumbBytes(ReaderWrapper.java:360)
at loci.formats.ChannelSeparator.openThumbBytes(ChannelSeparator.java:238)
at loci.formats.ReaderWrapper.openThumbBytes(ReaderWrapper.java:360)
at loci.formats.DimensionSwapper.openThumbBytes(DimensionSwapper.java:255)
at loci.formats.ReaderWrapper.openThumbBytes(ReaderWrapper.java:360)
at loci.formats.ReaderWrapper.openThumbBytes(ReaderWrapper.java:360)
at loci.formats.ReaderWrapper.openThumbBytes(ReaderWrapper.java:360)
at loci.formats.gui.BufferedImageReader.openThumbImage(BufferedImageReader.java:100)
at loci.plugins.in.ThumbLoader.loadThumb(ThumbLoader.java:147)
at loci.plugins.in.ThumbLoader.run(ThumbLoader.java:118)
at java.lang.Thread.run(Thread.java:750)
This issue has been mentioned on Image.sc Forum. There might be relevant details there:
https://forum.image.sc/t/hamamatsu-ndpi-files-will-not-open-with-bio-formats/69266/4
This issue has been mentioned on Image.sc Forum. There might be relevant details there:
https://forum.image.sc/t/error-generating-zstak-deep-zoom-images-ndpi-files-over-5gb/80800/6
A further report of this issue has been made on forum thread https://forum.image.sc/t/error-generating-zstak-deep-zoom-images-ndpi-files-over-5gb/80800/6
In this recent report it was consistent across multiple files, all over 5GB, but smaller diles and files larger than 7GB did not experience the problem. The original sample file from this issue was <300Mb so I don't think it is directly related to the file size.
The stack trace for the recent report is the same, but it always failed with Index 416640 out of bounds for length 414656
414656#java.lang.IndexOutOfBoundsException:
Index 416640 out of bounds for length 414656 at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
at java.base/java.util.Objects.checkIndex(Objects.java:372)
at java.base/java.util.ArrayList.get(ArrayList.java:459)
at loci.formats.services.JPEGTurboServiceImpl.getTile(JPEGTurboServiceImpl.java:297)
at loci.formats.services.JPEGTurboServiceImpl.getTile(JPEGTurboServiceImpl.java:251)
at loci.formats.in.NDPIReader.openBytes(NDPIReader.java:217)
at loci.formats.FormatReader.openBytes(FormatReader.java:919)
at loci.formats.ImageReader.openBytes(ImageReader.java:449)
The index out of bounds is occurring when trying to calculate which restartMarker to use in https://github.com/ome/bioformats/blob/develop/components/formats-bsd/src/loci/formats/services/JPEGTurboServiceImpl.java#L292-L298:
int mult = tileHeight / 8; // was restartInterval
int start = tileX + (tileY * xTiles * mult);
for (int row=0; row<tileHeight/8; row++) {
int end = start + 1;
long startOffset = restartMarkers.get(start);
long endOffset = in.length();
From debugging the original sample file from this issue, the number of restartMarkers for each series appears to correctly match those in the IFD metadata. This makes me suspect that it is the value of the restartInterval that is incorrect. From that original file, there are 5 series present and only the fourth series can be read correctly, the fifth series does not throw the same exception but it results in pixels values of all 0.
This issue has been mentioned on Image.sc Forum. There might be relevant details there:
https://forum.image.sc/t/error-generating-zstak-deep-zoom-images-ndpi-files-over-5gb/80800/8
Another report of this issue is seen on forum thread https://forum.image.sc/t/error-generating-zstak-deep-zoom-images-ndpi-files-over-5gb/80800/9
Two new sample files showing this issue are available at https://zenodo.org/record/7998534
With these sample files I was able to reproduce the exception using Bio-Formats 6.13.0:
java.lang.IndexOutOfBoundsException: Index: 52080, Size: 51832
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at loci.formats.services.JPEGTurboServiceImpl.getTile(JPEGTurboServiceImpl.java:297)
at loci.formats.services.JPEGTurboServiceImpl.getTile(JPEGTurboServiceImpl.java:251)
at loci.formats.in.NDPIReader.openBytes(NDPIReader.java:217)
This issue has been mentioned on Image.sc Forum. There might be relevant details there:
https://forum.image.sc/t/error-generating-zstak-deep-zoom-images-ndpi-files-over-5gb/80800/11