Pydantic error: extra fields not permitted for chain_type_kwargs

[keviddles]

Version: langchain-0.0.102

I am trying to run through the Custom Prompt guide here. Here's some code I'm trying to run:

from langchain.prompts import PromptTemplate
from langchain import OpenAI, VectorDBQA
prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.

{context}

Question: {question}
Answer in Italian:"""
PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)
chain_type_kwargs = {"prompt": PROMPT}
qa = VectorDBQA.from_chain_type(llm=OpenAI(), chain_type="stuff", vectorstore=docsearch, chain_type_kwargs=chain_type_kwargs)

I get the following error:

File /usr/local/lib/python3.10/dist-packages/langchain/chains/vector_db_qa/base.py:127, in VectorDBQA.from_chain_type(cls, llm, chain_type, **kwargs)
    125 """Load chain from chain type."""
    126 combine_documents_chain = load_qa_chain(llm, chain_type=chain_type)
--> 127 return cls(combine_documents_chain=combine_documents_chain, **kwargs)

File /usr/local/lib/python3.10/dist-packages/pydantic/main.py:342, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for VectorDBQA
chain_type_kwargs
  extra fields not permitted (type=value_error.extra)

[carterrees-entrata]

Seeing a similar error for ChatVectorDBChain

return cls( File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__ pydantic.error_wrappers.ValidationError: 1 validation error for ChatVectorDBChain qa_prompt extra fields not permitted (type=value_error.extra)

[anyuese]

same error 2 validation errors for ChatVectorDBChain chain extra fields not permitted (type=value_error.extra) retriever extra fields not permitted (type=value_error.extra

[carterrees-entrata]

What I have found is that I need to run pip install -U langchain everyday. And then make sure my code reflects the changes in the Langchain classes. These guys are moving so fast that the source code changes regularly. What worked a week ago might not work today. Its an exciting ride.

[annabechang]

similar error:

instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-xl", 
                                                      model_kwargs={"device": "cuda"})
                                                      

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[25], line 1
----> 1 instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-xl", 
      2                                                       model_kwargs={"device": "cuda"})

File ~/.conda/envs/py38/lib/python3.8/site-packages/langchain/embeddings/huggingface.py:102, in HuggingFaceInstructEmbeddings.__init__(self, **kwargs)
    100 def __init__(self, **kwargs: Any):
    101     """Initialize the sentence_transformer."""
--> 102     super().__init__(**kwargs)
    103     try:
    104         from InstructorEmbedding import INSTRUCTOR

File ~/.conda/envs/py38/lib/python3.8/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for HuggingFaceInstructEmbeddings
model_kwargs
  extra fields not permitted (type=value_error.extra)

[devosonder]

Worked for me. pip install -U langchain pip install langchain

[Amjad-AbuRmileh]

Seeing a similar error for ChatVectorDBChain

return cls( File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__ pydantic.error_wrappers.ValidationError: 1 validation error for ChatVectorDBChain qa_prompt extra fields not permitted (type=value_error.extra)

got any solution?

[Amjad-AbuRmileh]

similar error:

instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-xl", 
                                                      model_kwargs={"device": "cuda"})
                                                      

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[25], line 1
----> 1 instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-xl", 
      2                                                       model_kwargs={"device": "cuda"})

File ~/.conda/envs/py38/lib/python3.8/site-packages/langchain/embeddings/huggingface.py:102, in HuggingFaceInstructEmbeddings.__init__(self, **kwargs)
    100 def __init__(self, **kwargs: Any):
    101     """Initialize the sentence_transformer."""
--> 102     super().__init__(**kwargs)
    103     try:
    104         from InstructorEmbedding import INSTRUCTOR

File ~/.conda/envs/py38/lib/python3.8/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for HuggingFaceInstructEmbeddings
model_kwargs
  extra fields not permitted (type=value_error.extra)

Got any solution?

[Amjad-AbuRmileh]

similar error

pydantic.error_wrappers.ValidationError: 1 validation error for ChatVectorDBChain qa_prompt extra fields not permitted (type=value_error.extra)

updating langChain didn't help

[Amjad-AbuRmileh]

Version: langchain-0.0.102

I am trying to run through the Custom Prompt guide here. Here's some code I'm trying to run:

from langchain.prompts import PromptTemplate
from langchain import OpenAI, VectorDBQA
prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.

{context}

Question: {question}
Answer in Italian:"""
PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)
chain_type_kwargs = {"prompt": PROMPT}
qa = VectorDBQA.from_chain_type(llm=OpenAI(), chain_type="stuff", vectorstore=docsearch, chain_type_kwargs=chain_type_kwargs)

I get the following error:

File /usr/local/lib/python3.10/dist-packages/langchain/chains/vector_db_qa/base.py:127, in VectorDBQA.from_chain_type(cls, llm, chain_type, **kwargs)
    125 """Load chain from chain type."""
    126 combine_documents_chain = load_qa_chain(llm, chain_type=chain_type)
--> 127 return cls(combine_documents_chain=combine_documents_chain, **kwargs)

File /usr/local/lib/python3.10/dist-packages/pydantic/main.py:342, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for VectorDBQA
chain_type_kwargs
  extra fields not permitted (type=value_error.extra)

Got any solution?

[Jblauvs]

Also having this problem with HuggingFaceInstructEmbeddings, trying to pass in model_kwargs to load it onto the gpu.

HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-xl", model_kwargs = {'device': 'cuda:0'})

  File "/miniforge3/lib/python3.10/site-packages/langchain/embeddings/huggingface.py", line 102, in __init__
    super().__init__(**kwargs)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for HuggingFaceInstructEmbeddings
model_kwargs
  extra fields not permitted (type=value_error.extra)

Any fixes yet?

[DennisPeeters]

same with ConversationalRetrievalChain

[JoelHJames1]

Anyone was able to get this resolved?

[adrinanou]

have the same issue but with retrievalQA.from_chain_type(llm=llm,chain_type="map_reduce"....etc) validation error for map-reducedocumentschain extra fields not permitted(type=value_error.extra)

[levalencia]

Same here.

My code

  if 'chain' not in st.session_state:
        tfretriever = TFIDFRetriever.from_texts(
            ["My name is Luis Valencia",
            "I am 70 years old",
            "I like gardening, baking and hockey"])
        
        template = """
            Use the following context (delimited by <ctx></ctx>) and the chat history (delimited by <hs></hs>) to answer the question:
            ------
            <ctx>
            {context}
            </ctx>
            ------
            <hs>
            {chat_history}
            </hs>
            ------
            {question}
            Answer:
            """
        
        prompt = PromptTemplate(
            input_variables=["chat_history", "context", "question"],
            template=template,
            )
        
        st.session_state['chain'] = chain = ConversationalRetrievalChain.from_llm(llm, 
                                                                                vectordb.as_retriever(),
                                                                                memory=memory,
                                                                                  chain_type_kwargs={
                                                                                    "verbose": True,
                                                                                    "prompt": prompt,
                                                                                    "memory": ConversationBufferMemory(
                                                                                        memory_key="chat_history",
                                                                                        input_key="question"),
                                                                                })

Error:ValidationError: 1 validation error for ConversationalRetrievalChain chain_type_kwargs extra fields not permitted (type=value_error.extra) Traceback:

[OmiWakode]

Same issue with ConversationRetrievalChain.from_llm(llm = .., chain_type="map_reduce", question_promt=..., combine_prompt=....)

[levalencia]

same here https://github.com/hwchase17/langchain/issues/6635

[dromema]

extra fields not permitted (type=value_error.extra)

[plaban1981]

Same error PROMPT = PromptTemplate( template=prompt_template, input_variables=["context","question"] )

chain_type_kwargs = {"prompt": PROMPT}

qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", vectorstore=doc_store.as_retriever(), #chain_type_kwargs=chain_type_kwargs, return_source_documents=True)

##error ValidationError: 2 validation errors for RetrievalQA retriever field required (type=value_error.missing) vectorstore extra fields not permitted (type=value_error.extra)

[imrankh46]

I think this is so bad library, no one care about to update the documents.

[levalencia]

I think this is so bad library, no one care about to update the documents.

its open source, so you can actually contribute and write better documentation. Its a library with over 55K stars for a reason. :)

[SquirrelYe]

为我工作。 pip install -U langchain pip install langchain

It works for me, thanks a lot.

[ruze00]

I think this is so bad library, no one care about to update the documents.

its open source, so you can actually contribute and write better documentation. Its a library with over 55K stars for a reason. :)

I think he has a point however. It is not nice to give him a thumbs down for what is obviously a major issue in widespread adoption of the library. If companies cannot rely on stability, it is likely to fall out of favor for the next commercial offering which actually has support and consistency.

As it is, with 1270 (?) contributors it is much more likely to induce chaos if special efforts are not devoted to maintenance rather than constantly moving forward.

Don't shoot the messenger please. (Also, yes I think OP could have been more diplomatic).

[dosubot[bot]]

Hi, @keviddles! I'm Dosu, and I'm here to help the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

Based on the information provided, it seems that users are encountering Pydantic validation errors related to extra fields not being permitted for certain classes in the langchain library, such as VectorDBQA and HuggingFaceInstructEmbeddings. Several users have reported similar errors and have attempted to update and reinstall the library, but the issue persists. Some users have suggested running pip install -U langchain and pip install langchain as a potential solution. However, there hasn't been an official resolution or response from the repository maintainers.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, kindly let the LangChain team know by commenting on this issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and cooperation. Let me know if you have any further questions or concerns!

[kennethwork101]

Can someone please provide the version of langchian that works? I tried it on version 0.0.340 and it still failed.

llm = Ollama(model="codellama:7b", temperature=0.1, callback_manager=CallbackManager([StreamingStdOutCallbackHandler()])) a_chain = RetrievalQA.from_chain_type( llm, chain_type="map_reduce", retriever=vectordb.as_retriever(), return_source_documents=True, chain_type_kwargs={"prompt": QA_CHAIN_PROMPT}, )

---

return MapReduceDocumentsChain(

File "/home/kenneth/learning/venv/lib/python3.10/site-packages/langchain/load/serializable.py", line 97, in init super().init(**kwargs) File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 1 validation error for MapReduceDocumentsChain prompt extra fields not permitted (type=value_error.extra)

(venv) kenneth@kenneth-PC:~/learning/deeplearning/LangChain-Chat-with-Your-Data$ pip show langchain Name: langchain Version: 0.0.340

[dosubot[bot]]

@baskaryan Could you please help @kennethwork101 with this issue? They are encountering Pydantic validation errors related to extra fields not being permitted for certain classes in the langchain library, even after updating to version 0.0.340. They have provided additional details in their comment. Thank you!

[AlexCapchase]

Any solutions on this? I am getting the same error with BaseQAWithSourcesChain.from_chain_type(self.llm, retriever=pg_vector, **executor_config). Commenting the line of the configuration of the pydantic object to not have this line extra = Extra.forbid works, but I don't know if this is desired.

[siddhant250502]

similar error:

instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-xl", 
                                                      model_kwargs={"device": "cuda"})
                                                      

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[25], line 1
----> 1 instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-xl", 
      2                                                       model_kwargs={"device": "cuda"})

File ~/.conda/envs/py38/lib/python3.8/site-packages/langchain/embeddings/huggingface.py:102, in HuggingFaceInstructEmbeddings.__init__(self, **kwargs)
    100 def __init__(self, **kwargs: Any):
    101     """Initialize the sentence_transformer."""
--> 102     super().__init__(**kwargs)
    103     try:
    104         from InstructorEmbedding import INSTRUCTOR

File ~/.conda/envs/py38/lib/python3.8/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for HuggingFaceInstructEmbeddings
model_kwargs
  extra fields not permitted (type=value_error.extra)

Got any solution?

were you able to solve this error? @Amjad-AbuRmileh

[dosubot[bot]]

🤖

Based on the error message you provided, it seems like the VectorDBQA class is not expecting a chain_type_kwargs argument. The chain_type_kwargs argument is not defined in the VectorDBQA class, and Pydantic is raising an error because it does not allow extra fields that are not explicitly defined in the model.

The VectorDBQA class in the LangChain codebase is a subclass of BaseQAWithSourcesChain. It is used for question-answering with sources over a vector database. The class has the following attributes: vectorstore, k, reduce_k_below_max_tokens, max_tokens_limit, and search_kwargs. It does not have an explicit __init__ method, so it is instantiated with the arguments that correspond to these attributes.

In your code, you are trying to pass chain_type_kwargs as an argument to VectorDBQA.from_chain_type(), but this method does not expect such an argument. Instead, you should pass the prompt directly to the llm (language model) instance. Here's how you can modify your code:

from langchain.prompts import PromptTemplate
from langchain import OpenAI, VectorDBQA

prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.

{context}

Question: {question}
Answer in Italian:"""

PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)

# Instantiate the language model with the prompt
llm = OpenAI(prompt=PROMPT)

qa = VectorDBQA.from_chain_type(llm=llm, chain_type="stuff", vectorstore=docsearch)

In this modified code, the prompt is passed directly to the OpenAI instance when it is instantiated. Then, this llm instance is passed to VectorDBQA.from_chain_type().

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

• Missing "Prompt" in Together ai
• docs/extras/modules/chains/how_to/custom_chain.ipynb
• libs/langchain/langchain/chains/qa_with_sources/vector_db.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[isabellaaquino]

still getting the same error with RetrievalQA setting the chain_type as anything different than "stuff"

[Bella722]

It works for me that I used the most violent method possible. comment the error code line C:\Users\xxx.conda\envs\chatglm\Lib\site-packages\pydantic\v1\main.py line 341 values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data) # if validation_error: # raise validation_error