tantivy icon indicating copy to clipboard operation
tantivy copied to clipboard

Indexer panics

Open Barre opened this issue 1 year ago • 1 comments

Describe the bug

  • Inserting u64 values into a date field makes index writer panic called Result::unwrap()on anErr value: ErrorInThread("An index writer was killed.. A worker thread encountered an error (io::Error most likely) or panicked.")
  • Indexer panicked
  • Indexing should not panic.

Which version of tantivy are you using?

0.22.0

To Reproduce

https://github.com/Barre/tantivy_indexing_panic_date_reproduction/blob/master/src/main.rs

Barre avatar Nov 07 '24 10:11 Barre

It seems like caused by index_documents (here), as the error TantivyError::SchemaError("Expected a Date for field wrong_date") is returned, which terminates the thrd-tantivy-index-xxx then triggers the Drop call for index_writer_bomb

impl<D: Document> Drop for IndexWriterBomb<D> {
    fn drop(&mut self) {
        if let Some(inner) = self.inner.take() {
            inner.kill();
        }
    }
}
impl<D: Document> Inner<D> {
    fn kill(&self) {
        self.is_alive.store(false, Ordering::Relaxed);

and removing the ? here does helpful resolve the bug, maybe further optimization is needed ?

let _ = index_documents(
	mem_budget,
	index.new_segment(),
	&mut document_iterator,
	&segment_updater,
	delete_cursor.clone(),
);

pinylin avatar Feb 08 '25 15:02 pinylin