Adrien Guillo

The data is temporarily persisted on local disk until the index is successfully uploaded to S3. When indexing errors happen, we can resume from the local data. Most databases use...

> How often does this error occur? Yes, rarely. > And current implement can't provide durability of data because of the lack of replica. Yes. Our plan is to replicate...

This issue is fixed. The GC has moved to the janitor service.

François' PR was merged, closing.

The async API is nice but may be hard to use in practice without a native Quickwit client or support from log collectors. I'm in favor of a sync API...

@sophie-jeong, thanks for the report. Can you tell me the version of glibc your OS is running? @yangcancai, same question, which OS and version of glibc are you running?

Let's not merge splits from different sources. After the refactor, the merge key should be `(node_id, index_id, source_id)`.

> Currently, we have a split-store per source. This is inaccurate. Currently, there is [one split store indexing pipeline](https://github.com/quickwit-oss/quickwit/blob/main/quickwit-indexing/src/actors/indexing_pipeline.rs#L224). The layout you describe is also inaccurate. The current layout is...

You mentioned supporting something like `(hash(fields) mod n)` in another issue. This feature is usually called bucketing and is often implemented outside of partitioning (Hive, Spark, ...). I don't think...

Hi @gnufree, > I started 10 indexer nodes Unfortunately, as of today, Quickwit does not support multiple indexers working in parallel. You have to limit yourself to only one indexer....