annlite icon indicating copy to clipboard operation
annlite copied to clipboard

refactor: remove docarray dependency

Open JoanFM opened this issue 1 year ago • 2 comments

JoanFM avatar Jul 20 '23 13:07 JoanFM

Maybe we can define a simple pydantic model as the basic data structure:

class Document(BaseModel):
    id: str = Field(..., required=True)
    embedding: Union[nd.array, list[float]] = Field(..., required=True)
    meta_data: Dict = Field(..., default = {})

    def validator():
          # validate the shape of the embedding
          ...

class Query(BaseModel):
    ....

class Result(BaseModel):
    ....

Anyway, this idea can be in a new PR instead.

numb3r3 avatar Jul 21 '23 02:07 numb3r3

@numb3r3 I sitll want to do a change which is to set the key from which to extract the embedding, not always rely on embedding.

SInce my idea is to expose via DocArray, I will not use Pydantic for this since the validation would come from DocArray.

The idea here is that here it remains focused only on ANN search, nothing else

JoanFM avatar Jul 21 '23 07:07 JoanFM