java.io.IOException: block[ 0 ] already removed #386

[antonkulaga]

I am often getting "IOException: block[ 0 ] already removed " errors for zvi files, here is a example file that causes it https://drive.google.com/file/d/1U-d2MApVq3GROoEyWTPSA_Uz03AVA2lU/view?usp=sharing I use bioformats through pims (so also opened an issue https://github.com/soft-matter/pims/issues/386 , but I also saw it in other libs that use bioformats under the hood )

[dgault]

Hi @antonkulaga, thank you for opening the issue and providing a sample file. I have not been able to reproduce the issue when using Bio-Formats directly. Would you have a short piece of example code showing how you are accessing the images via pims?

Also do you know what other applications or libraries you have seen the issue with? Was it the same error in these cases?

[antonkulaga]

I use pims python library that is using latest bioformatcs under the hood (I also tried older versions without success). Here is the code https://github.com/antonkulaga/microscope_video_conversion The files that cause the exception are https://drive.google.com/file/d/1NB_rRLBPk1CNzArv0j8oQMFMwsRS3knc/view?usp=sharing and https://drive.google.com/file/d/1b_RNiWTxOF3K8xGsPXqrprjhUbHQkALd/view?usp=sharing Here is the screenshot of the error log

I have not been able to reproduce the issue when using Bio-Formats directly. Could you give me Java (or whatever JVM language you use) code snippet that did conversion for you so I can try it on my side as well?

[dgault]

Thanks @antonkulaga, I will try to reproduce with the samples provided and update this issue once I've learnt more

[dgault]

As a quick update, I can confirm that the issue is reproducible directly with Bio-Formats 6.6.1 using the sample files provided

[dgault]

Some initial debugging suggests that this is occuring when the POIFS tries to read the very first block which is null, the remaining blocks look to be populated and correct. Im not yet sure if this is a bug in the poifs or with the particular files.

The stack trace with showinf is as below:

Exception in thread "main" java.io.IOException: block[ 0 ] already removed
	at loci.poi.poifs.storage.BlockListImpl.remove(BlockListImpl.java:119)
	at loci.poi.poifs.storage.RawDataBlockList.remove(RawDataBlockList.java:56)
	at loci.poi.poifs.storage.BlockAllocationTableReader.fetchBlocks(BlockAllocationTableReader.java:217)
	at loci.poi.poifs.storage.BlockListImpl.fetchBlocks(BlockListImpl.java:151)
	at loci.poi.poifs.storage.RawDataBlockList.fetchBlocks(RawDataBlockList.java:56)
	at loci.poi.poifs.storage.SmallBlockTableReader.getSmallDocumentBlocks(SmallBlockTableReader.java:81)
	at loci.poi.poifs.filesystem.POIFSFileSystem.<init>(POIFSFileSystem.java:135)
	at loci.formats.services.POIServiceImpl.initialize(POIServiceImpl.java:86)
	at loci.formats.services.POIServiceImpl.initialize(POIServiceImpl.java:72)
	at loci.formats.in.ZeissZVIReader.initPOIService(ZeissZVIReader.java:229)
	at loci.formats.in.ZeissZVIReader.initVars(ZeissZVIReader.java:217)
	at loci.formats.in.BaseZeissReader.initFileMain(BaseZeissReader.java:136)