mindsdb icon indicating copy to clipboard operation
mindsdb copied to clipboard

[New Integration]: Integration with docarray part of jina.ai open source eco-system

Open rajivmehtaflex opened this issue 1 year ago • 1 comments

Is there an existing integration?

  • [X] I have searched the existing integrations.

Use Case

There is various reason for this integration. Mindsdb provide extraordinary support in create metadata. Metadata is a concept that associate dictionary with vector search. For e.g. on any ecommerce site if you search on domain (e.g. Electronics gadget ).Search result will show general information(price,brand etc..) as well domain specific information like. specification, this case is not apply for home decoration or books i.e. no need of specification. Instead this kind of search require different kind of metadata.Role of mindsdb is that it provide smart sort/filter by generating metadata as a inference.

for e.g. in search text box we can pass query like

  • "list of matching color for laces based on my [Brand-name] [Model number] shoes"
  • "List out most recommended washing powder with [Brand-name][Model number] washing machine "

Here we attach a shopping item with another shopping items.

Search part is handle by Jina.ai Meta data part is handle by MindsDB

Motivation

By implementation of such integration we are able to achieve

  • In-depth context based search
  • smart sort/filter (sort items based on color of my preference)

Here in various mid-range shopping sites/job search (Not from giants one). there is no way found items based on plain English. As per current scenario. They are mostly keyword based search. Because they don't know how to implement such solution. Through mindsdb and jina.ai philosophy it can be achieved.

Implementation

First part will be fetch data from docarray metadata to mindsdb --> Integration part. Second part mindsdb will generate metatdata and put in docarray back --> Document store part from jina ai. Third part smart search/sort/filter done by jina.ai

Anything else?

https://docarray.jina.ai/advanced/document-store/extend/

rajivmehtaflex avatar Oct 21 '22 14:10 rajivmehtaflex

Hi team, I would like to work on this issue and get assigned.

I would like to get a bit more clarity on the issue as I have used doc-array and MindsDB previously. Would like to know the expected outcomes through this integration.

Tagging: @ZoranPandovski @rajivmehtaflex

srini047 avatar Jan 19 '23 08:01 srini047

Hi @srini047, the community user proposed this issue, so we don't have too much info. Since you have experience with jina.ai which features we can integrate into MindsDB? Maybe integration to the document store?

ZoranPandovski avatar Feb 28 '23 12:02 ZoranPandovski

Closing this for now, If someone wants to work on this we will re-open

ZoranPandovski avatar Sep 29 '23 13:09 ZoranPandovski