milli
milli copied to clipboard
Remove and/or document the `ExternalDocumentsIds` struct
The ExternalDocumentsIds struct was a system that was designed to reduce the time spent removing external document ids into the FST containing them, this FST is a lookup table that points to the internal document id (u32). When we add a new value into an FST we must construct it from scratch, this hard/soft system is designed to amortize the cost by keeping the deleted/added values into the soft map, before merging the hard and the soft together at a given threshold.
This system is a little bit too complex and could maybe be documented more or removed entirely.
I would like to remove it from the codebase as described in https://github.com/meilisearch/milli/issues/76.