Recent ND2 files cannot be opened by bio-formats with NegativeArraySizeException

[aarpon]

Some recent ND2 files (NIS Elements version 5.2, maybe earlier; you can find an example here: https://drive.google.com/file/d/1nrLmO6KAfUVNf6Ie5YXupZZf5LQLgoBd/view?usp=sharing) fail opening in bio-formats (up to version 6.3.1) with the following exception:

(Fiji Is Just) ImageJ 2.0.0-rc-69/1.52p; Java 1.8.0_172 [64-bit]; Windows 10 10.0; 1798MB of 24263MB (7%)
 
java.lang.NegativeArraySizeException
	at loci.formats.in.NativeND2Reader.openBytes(NativeND2Reader.java:305)
	at loci.formats.DelegateReader.openBytes(DelegateReader.java:227)
	at loci.formats.ChannelFiller.openBytes(ChannelFiller.java:156)
	at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:229)
	at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:161)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
	at loci.formats.DimensionSwapper.openBytes(DimensionSwapper.java:233)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
	at loci.plugins.util.ImageProcessorReader.openProcessors(ImageProcessorReader.java:186)
	at loci.plugins.in.ImagePlusReader.readProcessors(ImagePlusReader.java:422)
	at loci.plugins.in.ImagePlusReader.readPlanes(ImagePlusReader.java:387)
	at loci.plugins.in.ImagePlusReader.readImage(ImagePlusReader.java:282)
	at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:243)
	at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:221)
	at loci.plugins.in.ImagePlusReader.openImagePlus(ImagePlusReader.java:116)
	at loci.plugins.in.Importer.readPixels(Importer.java:149)
	at loci.plugins.in.Importer.run(Importer.java:86)
	at loci.plugins.LociImporter.run(LociImporter.java:78)
	at ij.IJ.runUserPlugIn(IJ.java:230)
	at ij.IJ.runPlugIn(IJ.java:193)
	at ij.IJ.runPlugIn(IJ.java:182)
	at HandleExtraFileTypes.openImage(HandleExtraFileTypes.java:499)
	at HandleExtraFileTypes.run(HandleExtraFileTypes.java:72)
	at ij.IJ.runUserPlugIn(IJ.java:230)
	at ij.IJ.runPlugIn(IJ.java:193)
	at ij.IJ.runPlugIn(IJ.java:182)
	at ij.io.Opener.openWithHandleExtraFileTypes(Opener.java:516)
	at ij.io.Opener.openImage(Opener.java:370)
	at ij.io.Opener.openImage(Opener.java:242)
	at ij.io.Opener.open(Opener.java:109)
	at ij.io.Opener.openAndAddToRecent(Opener.java:291)
	at ij.plugin.DragAndDrop.openFile(DragAndDrop.java:192)
	at ij.plugin.DragAndDrop.run(DragAndDrop.java:159)
	at java.lang.Thread.run(Thread.java:748)

[dgault]

Thanks for reporting the bug and providing a sample file. I was able to reproduce the exception using the latest Bio-Formats release (6.3.1) and neither of the ND2 options (chunkmap or native lib) had any impact.

Would it be possible to make the provided sample file available publicly under the CC-BY license?

[aarpon]

Hi,

sorry for the delay: I had to double-check with the owner of the file. The file can be made available publicly under the terms of the CC-BY license. Do I have to do something about it myself?

One more thing: we cropped the image in NIS Elements and the smaller file ( https://drive.google.com/file/d/1_qR5lagexpVq0rgkHvRAFhopBxPCL6ok/view?usp=sharing) could be opened in Fiji successfully.

Thanks, a2

On Fri, Mar 6, 2020 at 2:58 PM David Gault [email protected] wrote:

Thanks for reporting the bug and providing a sample file. I was able to reproduce the exception using the latest Bio-Formats release (6.3.1) and neither of the ND2 options (chunkmap or native lib) had any impact.

Would it be possible to make the provided sample file available publicly under the CC-BY license?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ome/bioformats/issues/3517?email_source=notifications&email_token=AAJLKO43NYQW7O4UMA7RY5LRGD6RTA5CNFSM4LCEI4MKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOBN6SA#issuecomment-595779400, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJLKO2YKY7KOJAAL2OAIZLRGD6RTANCNFSM4LCEI4MA .

[melissalinkert]

I believe https://forum.image.sc/t/error-while-opening-nd2-file/44700 is another example of this issue. The problem appears to be specific to files with multiple channels that are stored interleaved when the total pixel count for a plane (X * Y * channels * bits) exceeds 2 GB. Opening smaller tiles is a work-around in the meantime, but we should be able to fix this for the case when X * Y * bits is less than 2 GB.

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/error-while-opening-nd2-file/44700/3

[dgault]

An additional report of this issue comes from forum thread https://forum.image.sc/t/large-nd2-file-doesnt-open-negativearraysizeexception/54897. A new sample file reproduces the exception with Bio-Formats 6.6.1

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/large-nd2-file-doesnt-open-negativearraysizeexception/54897/5

[melissalinkert]

The file from https://forum.image.sc/t/error-while-opening-nd2-file/44700 no longer has this problem; https://github.com/openmicroscopy/data_repo_config/pull/590 adds it to the test repo.

I was not able to find the other files referenced here in inbox, though, so leaving this open until we can confirm that they are working as well.

[dgault]

A further report of this issue has been received for stitched files on the forum thread: https://forum.image.sc/t/large-nd2-file-doesnt-open-negativearraysizeexception/54897/10