SunisDown

Thanks for you replay @fulmicoton , let's ignore my own use case and focus on the endpoint. My use case is not mentioned in this issue. I didn't open this...

> Non-replicated PushAPI + refresh=true. More or less the same trade-off as stateless PushAPI, but internally uses the plumbing of Non-replicated PushAPI, triggers a commit and wait for it. Noted...

> In quickwit, the data is written to disk but the log is not replicated yet. "The data is written to disk" The plan to add replication there is closer...

Share my expectations for Quickwit's distributed storage. - The distributed storage should serve both indexer and searcher nodes 1. for Indexer, can write data to the distributed storage more frequently,...

>The selector coudl be k % N looks good enough to select a subset of kafka partition. > Publishing a split requires interacting with a metastore (today write a file...

> The selector coudl be k % N can we use something like `consistent hash`? If we use `k % N`, when there's some infra changes like remove/add in indexer...

> If a rebalance occurs while split creation is in progress and partition assignments change -> the in progress split will have to be invalidated and restarted on the latest...

same issue. On macOS the response data is `csrf_failure`

@generall MmapVectors or the index already use `MADV_RANDOM` while opening the file. I'm not sure how can we make the file as `WILLNEED`. do you want to set as `WILLNEED`...

I have made some simple testing on the madivse as `WILLNEED`. But seems that the performance was even worse when I enable `WILLNEED`. ```rust fn load_from_file(path: &Path) -> OperationResult {...