annlite icon indicating copy to clipboard operation
annlite copied to clipboard

support upload/download model to/from hubble

Open jemmyshin opened this issue 2 years ago • 1 comments

Since we move to jcloud deployment, it's necessary to support uploading/downloading PCA/PQ model to/from Hubble.

Thus, we need to implement these APIs:

self._projector_codec.upload(artifact='...') self._projector_codec.download(artifact='...')

The artifact is determined by users and should be consistency throughout the whole pipeline. And also should be passed to jcloud.yaml.

jemmyshin avatar Aug 25 '22 06:08 jemmyshin

What's the API design in annlite? Is this what we are expected:

def save_model(self, name: Union[str, PathLike]):
    if isinstance(name, str):
        # save to hubble with the given name `name`
    else:
        # save to local path
    

The user journal looks like:



index = Annlite(...)

index.train(...)

# upload the model parameters 
index.save_model('pca128_pq64')

# load the model  parameters 
index = Annlite(..., model_path='pca128_pq64')
index.index(...)
index.search(...)

numb3r3 avatar Aug 26 '22 03:08 numb3r3