quickwit icon indicating copy to clipboard operation
quickwit copied to clipboard

Ingest V2 misses commit force

Open fulmicoton opened this issue 5 months ago • 0 comments

Ideally, we would like quickwit to be ready to ingest documents right after it responded to a create index request. In practise this is not the case for ingest v1.

When we create a new index, if we experience the cooldown experienced in #4416, indexer node won't be informed right away about the addition of the new index. Even before, that information was sent asynchronously, and there was a race condition, although much less likely to occur.

In Ingest V2, this is not a problem but another problem arises : commit=force is not supported yet.

My hunch: We can assume ingest v1 to be deprecated in the next version and focus on implementing commit=force in the next version. We can then remove the sleep_after clause in rest integration test, introduce in #4416

fulmicoton avatar Jan 22 '24 07:01 fulmicoton