azure-open-ai-embeddings-qna icon indicating copy to clipboard operation
azure-open-ai-embeddings-qna copied to clipboard

embedding model not detected

Open cpavel opened this issue 1 year ago • 5 comments

Keep running into the issue of my embedding model not being detected by the app. Error details below. I made sure I have the deployment as per name below and model as per name below. It is right next to the LLM model (GPT-4).

LLM is working!

Embedding model is not working. Please check you have a deployment named "text-embedding-ada-002" for "text-embedding-ada-002" model in your Azure OpenAI resource https://i*********.openai.azure.com/. Then restart your application.

Traceback (most recent call last): File "/usr/local/src/myscripts/OpenAI_Queries.py", line 31, in check_deployment llm_helper.embeddings.embed_documents(texts=["This is a test"]) File "/usr/local/lib/python3.9/site-packages/langchain/embeddings/openai.py", line 258, in embed_documents return self._get_len_safe_embeddings(texts, engine=self.document_model_name) File "/usr/local/lib/python3.9/site-packages/langchain/embeddings/openai.py", line 194, in _get_len_safe_embeddings encoding = tiktoken.model.encoding_for_model(self.document_model_name) File "/usr/local/lib/python3.9/site-packages/tiktoken/model.py", line 51, in encoding_for_model raise KeyError( KeyError: 'Could not automatically map ruocco-ada to a tokeniser. Please use tiktok.get_encoding to explicitly get the tokeniser you expect.'

cpavel avatar Aug 11 '23 13:08 cpavel