lucene-solr icon indicating copy to clipboard operation
lucene-solr copied to clipboard

LUCENE-9409: Truncation can also cause IndexOutOfBoundsException.

Open jpountz opened this issue 4 years ago • 4 comments

Expect IndexOutOfBoundsException when opening indices with truncated files.

jpountz avatar Jun 18 '20 07:06 jpountz

Sorry, I'm against this change. The test is broken. It looks like we are willing to make bad tradeoffs in order to deliver CorruptIndexException and only CorruptIndexException if anything goes wrong. Fix the test instead!

A side-effect of this is that we can no longer verify checksums of the meta file before checking the length of other files

This is seriously the wrong tradeoff: let's fix the test instead. If we unexpectedly hit EOF, EOFException is the correct exception. If an index is out of bounds, IndexOutOfBoundsException is the correct exception.

rmuir avatar Jun 21 '20 10:06 rmuir

I like the CorruptIndexException because it tells me that the problem is that the file got altered after being written, while I would otherwise wonder if there is a bug in Lucene. As an alternative, would it work better for you if we called retrieveChecksum(IndexInput) before the try block, and then again with the length (retrieveChecksum(IndexInput, long)) after the try block once the checksum of the meta file has been validated?

jpountz avatar Jun 22 '20 13:06 jpountz

I repurposed this PR to instead make the test expect out-of-bounds exceptions. Does it look better to you @rmuir @uschindler ?

jpountz avatar Aug 11 '20 17:08 jpountz

I am fine to fix the test. Sure you have to first figure out why the index is out of bounds, and the exact exception may be misleading, but that's actually what's happening here. If you want other exceptions, another fix would be to enforce the IO layer to have a meaningful exception and implement it for all directory implementations.

uschindler avatar Aug 11 '20 17:08 uschindler