eland icon indicating copy to clipboard operation
eland copied to clipboard

Add CLI for deploying built-in models

Open maxjakob opened this issue 1 year ago • 4 comments

In the context of integration tests and generally in cluster setup automation, the eland_import_hub_model CLI is a great tool to get HuggingFace models deployed in Elasticsearch. There is no a comparable tool for built-in models (most notably ELSER and E5). Users have to do this themselves using with 4 different API calls and a wait condition (example). Adding a CLI tool eland_deploy_builtin_model that works in a similar way (e.g. --model-id .elser_model_v2 --url http://elasticsearch:9200 --start) would reduce the room for error, reduce friction when using built-in models and in turn increase their adoption.

maxjakob avatar May 14 '24 12:05 maxjakob

Should this be in eland or in the main client libs? It's just some API calls.

joshdevins avatar May 14 '24 15:05 joshdevins

The API calls should definitely be in the vectorstore helpers in the client. That will directly improve snippets like https://www.elastic.co/search-labs/tutorials/search-tutorial/semantic-search/elser-model#deploying-the-elser-model which can easily timeout and leave you in a bad state that is difficult to recover from (eg. model deployed but no ingest pipeline).

But it seems to me that there's also value in exposing this through Eland? It would be nice to have a single command to upload both E5 and ELSER in my opinion.

pquentin avatar May 15 '24 05:05 pquentin

Implementing this in the client's helpers module works. Then we also have a simple programmatic way to do this in Python.

Additionally, I think it's also important to have a CLI for this. This could of course also be through the client package. But since users are already used to deploying HF models with Eland I think it's a good fit to add more model deployment tooling here (also).

maxjakob avatar May 15 '24 09:05 maxjakob

This would also remove the necessity for these large code blocks in our blog posts: https://www.elastic.co/search-labs/blog/serverless-semantic-search-with-elser-in-python#download-and-deploy-elser-model

maxjakob avatar Jul 29 '24 08:07 maxjakob