Henry Lindeman
Henry Lindeman
@austintlee is this PR still alive?
Looks like the env var gets picked up in [this](https://github.com/deepjavalibrary/djl/blob/edd2e65694d5514dac0e11b3bce9336ed791d06d/api/src/main/java/ai/djl/util/Utils.java#L415) method. Any chance the OpenSearch security manager has problems with reading env vars?
What logs do you get near the 'downloading pytorch' bit? I also notice your OS is RHEL docker - are you running in a docker container?
I think going the connector route makes the most sense to me. Yes, it's currently designed exclusively for doing external model inferences, but at its heart it's just a primitive...
Oh, how do they do it in notifications?
looks like maybe https://github.com/huggingface/transformers/issues/20815 is related? Is mxbai-rerank based on Deberta do you know? If so try setting `model_type='deberta'` in the make_model_config_json fn call
I ran into this when integrating with the weaviate client: ```python import weaviate import ray ray.init() ``` python 3.11 // macos 14.4.1
> We are also planning of standardizing the local model predict output with the same format as the remote model predict output, then it will ease the pain of post...
Gonna implement this as a new ml-algorithm / function name: `TEXT_SIMILARITY`. Cross-encoders are one technique to do this, but not the only one. Essentially it's just defined as (text1, text2)...
A bunch of requests have come in for remote reranking models too, so I'll make sure that connectors can deal with TextSimilarityDatasets appropriately as well