eland icon indicating copy to clipboard operation
eland copied to clipboard

[NLP] Add model compatibility check mode to eland_import_hub_model.py

Open davidkyle opened this issue 2 years ago • 0 comments

The best way to check if a model is compatible with Elasticsearch is to run the eland_import_hub_model.py script but this requires the user to configure the Elasticsearch connection settings and possibly even spin up an Elasticsearch cluster in order to get past the connection test.

Add a command option to the script that will download the model and check it is compatible with Elastic without uploading the model to Elasticsearch. This is a convenience to quickly check if a model is compatible with minimal configuration.

Invoking the script might look like:

    eland_import_hub_model \
      --is-compatible
      --hub-model-id 'sentence-transformers/msmarco-MiniLM-L-12-v3 \
      --task-type text_embedding

davidkyle avatar Sep 12 '23 10:09 davidkyle