[v1.6] Implement vector search experimental feature

[curquiza]

trafficstars

Hello dear community

This library currently handles an experimental feature present in Meilisearch (the engine) since v1.3.0: the vector search.

v1.6.0 of Meilisearch is released on 15/01/2024, and this new version introduces additions to vector search, but also changes related to these experimental features.

This means we need to update the repository code base according to the new changes.

Unfortunately, the internal Meili team cannot afford to spend time implementing the changes, which is why we opened this issue to the community.

If this issue is not implemented before 15/01/2024, don’t worry 😊 We will do a clear communication saying the library cannot work with the experimental v1.6.0 vector search for the moment until someone takes it! Of course, even if we don’t implement it, users will still be able to use all the introduced stable features.

So no pressure on the community, just to let you know and be transparent with our choices.

Thank you in advance for your help ❤️

👇 Below you will find what we expect to be done to make the library up-to-date with the vector search feature in Meilisearch v1.6.0

---

Refer to the changelogs to have a global view of the changes: https://github.com/meilisearch/meilisearch/releases/tag/v1.6.0

Additions

• [ ] Ability to receive the embedders setting when calling the /indexes/:index_uid/settings route (GET/PATCH), so the methods to get and update global settings of an indexes should be adapted.
• [ ] Introduction of new routes
  • [ ] Create a new method to get the settings by calling GET /indexes/:index_uid/settings/embedders
  • [ ] Create a new method to update the settings by calling PATCH /indexes/:index_uid/settings/embedders
  • [ ] Create a new method to reset the settings by calling DELETE /indexes/:index_uid/settings/embedders
• [ ] Introduction of a new parameter during the search: hybrid. It can take two fields: semanticRatio and embedder.
• [ ] ⚠️ Tests should be added

Changes

Usage changes introduced in v1.6.0 should not impact the code base of the library. Let me know on the issue if we are wrong.

• [ ] Uncomment tests regarding vector search in search_test.dart following https://github.com/meilisearch/meilisearch-dart/pull/368 and make it work with the new behavior
• [ ] Update the README section to remove the mention of vector search issue

More instructions

👉 If you want details about how the feature works, refer to this: https://meilisearch.notion.site/v1-6-Hybrid-Search-Embedders-ea42c82f90cc4bc0be1eeb917c1118c8

[curquiza]

Since v1.7.0 OpenAI models now accepts dimensions field. The implementation should reflect this

[curquiza]

Since v1.8.0

• new models have been introduced
  • Ollama model
  • Generic REST embedder
• a new embedder settings: distribution

Check changelogs for details

[curquiza]

Since v1.9.0

• Breaking: _vectors no longer returned in documents (by default). Use retrieveVectors during the search to see it -> this library should accept the retrieveVectors search parameter at search

[brunoocasali]

Since v1.10.0

Usage: https://meilisearch.notion.site/v1-10-AI-search-changes-737c9d7d010d4dd685582bf5dab579e2?pvs=74

• Rest API
  • Removed parameters: query, inputField, inputType, pathToEmbeddings and embeddingObject.
  • Replaced by request and response
  • New parameter: headers
• Add url parameter to the OpenAI embedder
• dimensions is now available as an optional parameter for ollama embedders.