List of improvements on the nested field post v0.27.0 RC0

[irevoire]

trafficstars

• [ ] Remove a now useless callback in the transform: Originally posted by @Kerollmops in https://github.com/meilisearch/milli/pull/458#discussion_r845274181
• [x] Use the smartstring crate to store the primary key in the transform. => It should improve the performance but more importantly: it'll reduce memory consumption. #493
• [x] See if we can improve the error message for the indexing of geopoints => we were able to show the complete object before. Now that's impossible but we could specify what we didn't find (latitude or longitude) => #523
• [x] Skip all the flattening if the object do not need it => rewrite a specialized version of oxidized-json-checker that measure the « depth » in the value (actually it's not the depth because we can have 0 object and 1 array). Maybe we could even create something even easier and faster. #494
• [x] Flatten by fields instead of by document => https://github.com/meilisearch/milli/pull/514
• [ ] Maybe we could flatten (and thus index) only the fields that are actually in use (ie in any settings)
• [ ] Maybe we could fasten the search by creating a « true » db for the attribute to [ retrieve / highlight / crop ] with all the flattened field in it at the indexing time. Not sure about this one.
• [ ] Try to make the flatten-serde-json crate works with obkv directly
• [ ] See if there is something to do with bumpalo to make all the small allocations a lot faster when we flatten document and reuse the same buffer between each loops (basically what we're doing with the obkv currently)

[curquiza]

Also, we could add some benchmarks to bench a dataset with nested fields 😇 See: https://github.com/meilisearch/milli/issues/500

[Kerollmops]

I was trying to take this small one Remove a now useless callback in the transform, but it looks like you use it now, am I right?

[irevoire]

Yes but now it doesn't really represent what it was :grimacing: https://github.com/meilisearch/milli/pull/458#discussion_r845274181

[irevoire]

Everything has been tested