grenad
grenad copied to clipboard
meilisearch
Tools to sort, merge, write, and read immutable key-value pairs :tomato:
We use grenad in Meilisearch, and we often have the _too many open files (os error 24)_ error, which stops the whole indexation. I want to propose a change in...
In this PR, I made the `Sorter` able to sort in parallel in different ways that I documented: - [x] Make the sorter able to sort the keys by using...
Currently, the merge function is anonymous. It is not very practical when you are refactoring code. I mean, typing thing is much better and makes the work easier. The code...
The Sorter is doing a lot of operations to ensure the values are merged in the correct order. I thought about a simple way to speed the sort operation a...
This PR exposes two cargo features to enable glidesort as the sort algorithm for the grenad Sorter: - `glidesort-stable`: Use glidesort to stable sort in the Sorter instead of using...
It could be useful to be able to get the latest value inserted in a sorter. There can be small lifetime issues but we can maybe just clone the value...
We use grenad in Meilisearch, and when we benchmark and flamegraph the indexing process, which is intensively utilizing this library, we can see that 12-15% of the time is passed...
https://github.com/meilisearch/grenad/blob/f6808d8b6f37e0eae167dad89d08edb15652b97d/src/compression.rs#L90-L99
Why did you choose an u64 for the length? Isn't it too much? Moreover, if we reach the limit of an u64 we'll try to load more than `150 000`...
Metadata
Owner
Metadata
Tools to sort, merge, write, and read immutable key-value pairs :tomato: