FlowSightReader: Tried to read past end of IFD data

[jburel]

Attempting to convert cif files associated to https://www.ebi.ac.uk/biostudies/BioImages/studies/S-BIAD452?query=S-BIAD452 during ABis workshop to help one of the participants who wants to access specific IDR data and BIA data

Caused by: java.lang.IndexOutOfBoundsException: Tried to read past end of IFD data
	at loci.formats.in.FlowSightReader$1.next(FlowSightReader.java:451)
	at loci.formats.in.FlowSightReader$1.next(FlowSightReader.java:368)
	at loci.formats.in.FlowSightReader.openGreyscaleBytes(FlowSightReader.java:468)
	at loci.formats.in.FlowSightReader.openBytes(FlowSightReader.java:302)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:348)
	at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:229)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:348)
	at loci.formats.MinMaxCalculator.openBytes(MinMaxCalculator.java:269)
	at loci.formats.MinMaxCalculator.openBytes(MinMaxCalculator.java:260)
	at com.glencoesoftware.bioformats2raw.Converter.getTile(Converter.java:1069)
	at com.glencoesoftware.bioformats2raw.Converter.processChunk(Converter.java:1179)
	at com.glencoesoftware.bioformats2raw.Converter.lambda$saveResolutions$4(Converter.java:1387)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)

[jburel]

It looks like we will have to recombine the 3 parts sample0_part1_of_3.cif, sample0_part2_of_3.cif, sample0_part3_of_3.cif

[rishiraj88]

In FlowSightReader.java:449, consider changing the implementation to: @Override public Short next() { /* if (! hasNext()) throw new IndexOutOfBoundsException("Tried to read past end of IFD data");*/ if (! hasNext()) return (Short) 0; loaded = false; return value; }

[melissalinkert]

Four files files uploaded to inbox/gh-4043 in case further investigation is needed.

There are two versions of sample0_part1_of_3.cif. The first time I downloaded, the result was a ~900MB file; there were no transfer errors that I could see. I could reproduce the exception above with 6.14.0 and showinf sample0_part1_of_3.cif -series 32296 (the last series in the file). The second time I downloaded (a few hours later), the result was a ~2.6GB file and I was unable to reproduce the error with 6.14.0 and showinf sample0_part1_of_3_retry.cif -series 99999 (the last series in the retried download).

Both sample0_part2_of_3.cif and sample0_part3_of_3.cif downloaded as ~2.5GB files, and showinf on the first and last series worked without error. All 3 >2GB files have 100000 series.

Before we investigate further, does downloading the files again solve the problem? For reference, the md5sum of the 3 working files that are >2GB:

31fef6157af7b99990b3aff9702ccdc5  sample0_part1_of_3_retry.cif
a212c816330562a5dc546717f99f4cde  sample0_part2_of_3.cif
6311088bf80e16733fd0642730f97938  sample0_part3_of_3.cif

and the md5sum of the failing file:

d52447a26902973649e4c36d3fb09efc  sample0_part1_of_3.cif

[jburel]

Thanks @melissalinkert I will try. It could be a download problem