Using LLama Embedings still rely on OpenAI key

[iiAnthony]

System Info

Runs under jupyterlab in docker platform : Linux-5.4.72-microsoft-standard-WSL2-x86_64-with-glibc2.29 python : 3.8.5

Who can help?

No response

Information

• [ ] The official example notebooks/scripts
• [X] My own modified scripts

Related Components

• [ ] LLMs/Chat Models
• [X] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [ ] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

When trying to use a Llama model on local documents I have the following very basic piece of code :

from langchain.document_loaders import DirectoryLoader

loader = DirectoryLoader('./', glob="**/*.yml", show_progress=True)
local_model_path = './models/ggml-gpt4all-l13b-snoozy.bin'

from langchain.embeddings import LlamaCppEmbeddings

llama = LlamaCppEmbeddings(model_path=local_model_path)

from langchain.indexes import VectorstoreIndexCreator
index = VectorstoreIndexCreator(embeddings=llama).from_loaders([loader])
index.query("what are the CORE variables ?")

No specific requirement of any OpenAI tool, but I have the error below :

AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | 
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
<ipython-input-12-26dba7aded6b> in <module>
      3 
      4 from langchain.indexes import VectorstoreIndexCreator
----> 5 index = VectorstoreIndexCreator(embeddings=llama).from_loaders([loader])
      6 index.query("what is the LHYFE variables ?")

/usr/local/lib/python3.8/dist-packages/pydantic/main.cpython-38-x86_64-linux-gnu.so in pydantic.main.BaseModel.__init__()

/usr/local/lib/python3.8/dist-packages/pydantic/main.cpython-38-x86_64-linux-gnu.so in pydantic.main.validate_model()

/usr/local/lib/python3.8/dist-packages/pydantic/fields.cpython-38-x86_64-linux-gnu.so in pydantic.fields.ModelField.get_default()

/usr/local/lib/python3.8/dist-packages/pydantic/main.cpython-38-x86_64-linux-gnu.so in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for OpenAIEmbeddings
__root__
  Did not find openai_api_key, please add an environment variable `OPENAI_API_KEY` which contains it, or pass  'openai_api_key' as a named parameter. (type=value_error)
​```

Is there any specific configuration I missed out ? Many thanks for your kind help.

Expected behavior

I would have expected it to use the model stated in the code without any need for an OpenAI account.

[imeckr]

VectorstoreIndexCreator.query methods uses RetrievalQA which requires a LLM to be provided, if not provided it defaults back to OpenAI llm thats why you are getting this exception. You could pass the llama llm to query method, something like this

from langchain.llms import LlamaCpp
llama_llm = LlamaCpp(model_path="./ggml-model-q4_0.bin")

index.query("what are the CORE variables ?", llm=llama_llm)

[decadance-dance]

@imeckr, you are right. But I have the same bug and this error occurs while creating of VectorstoreIndexCreator.

Code:

from langchain.embeddings import LlamaCppEmbeddings
from langchain.indexes import VectorstoreIndexCreator

llama_emb = LlamaCppEmbeddings(model_path=model_path)
index = VectorstoreIndexCreator(embeddings=llama_emb).from_loaders([loader])

Error:

ValidationError: 1 validation error for OpenAIEmbeddings
__root__
  Did not find openai_api_key, please add an environment variable `OPENAI_API_KEY` which contains it, or pass  `openai_api_key` as a named parameter. (type=value_error)

That is, I even don't run query method.

[imeckr]

@decadance-dance I see a typo in your code:

index = VectorstoreIndexCreator(embeddings=llama_emb).from_loaders([loader]) => index = VectorstoreIndexCreator(embedding=llama_emb).from_loaders([loader])

[decadance-dance]

@imeckr Really, my bad, thanks.

[ratnesh1729]

The query from index.query(...) still wants me to use OpenAI API key? I am not sure how to avoid this. Looking for some ideas

[jeffxtang]

The query from index.query(...) still wants me to use OpenAI API key? I am not sure how to avoid this. Looking for some ideas

As @imeckr said, and I just confirmed this myself:

from langchain.llms import LlamaCpp llama_llm = LlamaCpp(model_path="./ggml-model-q4_0.bin")

index.query("what are the CORE variables ?", llm=llama_llm)

[dosubot[bot]]

Hi, @iiAnthony,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. It seems that some users have encountered a similar bug and received assistance in resolving it by passing the llama llm to the query method or correcting a typo in the code.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and cooperation.