open-text-embeddings
open-text-embeddings copied to clipboard
Make sure to set the trust_remote_code
When using then open source models like - jinaai/jina-embeddings-v2-base-en, it throws an error for setting the trust_remote_code to True
Please update the code for handling the same using the below option as mentioned in https://github.com/langchain-ai/langchain/issues/6080
if "jina" in model_name:
tokenizer = AutoModel.from_pretrained(model_name, trust_remote_code=True)
Thanks for open the issue.
The jina-embeddings models are not officially supported yet. I am still waiting for the release of the jina-embeddings-v2-large then look into it.
By the way, did you try to pull the model locally and run it without trust_remote_code?
I just deploy the jinaai/jina-embeddings-v2-base-en to Modal, setting the trust_remote_code is not needed. You might interested in the latest updates on cloud deployment at https://github.com/limcheekin/open-text-embeddings/tree/main?tab=readme-ov-file#%EF%B8%8F-cloud-deployment.