milli icon indicating copy to clipboard operation
milli copied to clipboard

Remove and/or document the `ExternalDocumentsIds` struct

Open Kerollmops opened this issue 3 years ago • 1 comments
trafficstars

The ExternalDocumentsIds struct was a system that was designed to reduce the time spent removing external document ids into the FST containing them, this FST is a lookup table that points to the internal document id (u32). When we add a new value into an FST we must construct it from scratch, this hard/soft system is designed to amortize the cost by keeping the deleted/added values into the soft map, before merging the hard and the soft together at a given threshold.

This system is a little bit too complex and could maybe be documented more or removed entirely.

Kerollmops avatar Jun 13 '22 13:06 Kerollmops

I would like to remove it from the codebase as described in https://github.com/meilisearch/milli/issues/76.

Kerollmops avatar Sep 27 '22 15:09 Kerollmops