LLMMathChain to allow ChatOpenAI as an llm

[cnndabbler]

1. Cannot initialize match chain with ChatOpenAI LLM

llm_math = LLMMathChain(llm=ChatOpenAI(temperature=0))

---

ValidationError Traceback (most recent call last) Cell In[33], line 1 ----> 1 llm_math = LLMMathChain(llm=ChatOpenAI(temperature=0))

File ~/anaconda3/envs/gpt_index/lib/python3.8/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.init()

ValidationError: 1 validation error for LLMMathChain llm Can't instantiate abstract class BaseLLM with abstract methods _agenerate, _generate, _llm_type (type=type_error)

2. Works ok with OpenAI LLM

llm_math = LLMMathChain(llm=OpenAI(temperature=0))

[apelliciari]

same error with SQLDatabaseChain + ChatOpenAI

    db = SQLDatabase.from_uri("sqlite://...")
    llm = ChatOpenAI(temperature=0)
    db_chain = SQLDatabaseChain(llm=llm, database=db, verbose=True)

[benheckmann]

@cnndabbler Are you currently working on this? Otherwise, I would take on this issue.

Apart from changing the typing of LLMMathChain.llm to allow for a BaseChatModel, I would also suggest changing the default prompt.

Currently, it lets the language model choose whether to use python code or not, which works with text-davinci-003, but leads to an invalid answer with gpt-3.5-turbo (something like "Sorry, as a Language Model I cannot...."), when running the example from the docs. I would modify it to ask the model only for python code. I cannot imagine any problem that a language model can confidently solve by itself that it cannot solve via python code.

[GiovanniGrotto]

Same error trying to use gpt-3.5-turbo with VectorStoreRouterToolkit.

pydantic.error_wrappers.ValidationError: 1 validation error for VectorStoreRouterToolkit
Can't instantiate abstract class BaseLLM with abstract methods _agenerate, _generate, _llm_type (type=type_error)

llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=temperature)
router_toolkit = VectorStoreRouterToolkit(
    vectorstores=[...],
    llm=llm,
)

But it works with davinci-003:

 llm = OpenAI(temperature=temperature)

[alanpog]

same error with SQLDatabaseChain + ChatOpenAI

    db = SQLDatabase.from_uri("sqlite://...")
    llm = ChatOpenAI(temperature=0)
    db_chain = SQLDatabaseChain(llm=llm, database=db, verbose=True)

Actually it was fixed for SQLDatabaseChain last week in this commit: b1c4480d7cdf0b86d204641fc126188929a46439 .

However, it still requires a BaseLLM in LLMMathChain as reported by cnndabbler .

[jcomfort4]

I was trying to query a VectorstoreIndexCreator() using a custon llm from hugging face based on "OpenAssistant/oasst-sft-1-pythia-12b" and ran into a similar issue. I think the fix here would be something like this commit: https://github.com/hwchase17/langchain/commit/b1c4480d7cdf0b86d204641fc126188929a46439

Any guidance on what needs to be changed in LLMChain?

ValidationError: 1 validation error for LLMChain
llm
  Can't instantiate abstract class BaseLanguageModel with abstract methods agenerate_prompt, generate_prompt (type=type_error)

Doing something like this...

tokenizer = AutoTokenizer.from_pretrained("OpenAssistant/oasst-sft-1-pythia-12b")

model = AutoModelForCausalLM.from_pretrained("OpenAssistant/oasst-sft-1-pythia-12b")

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    max_length=1024
)

local_llm = HuggingFacePipeline(pipeline=pipe)
llm_chain = langchain.LLMChain(
    prompt=PromptTemplate(template=template, input_variables=["question"])
    llm=local_llm
)

index = VectorstoreIndexCreator().from_loaders([loader])
index.query(query, llm=llm_chain)

[cnndabbler]

@cnndabbler Are you currently working on this? Otherwise, I would take on this issue.

Apart from changing the typing of LLMMathChain.llm to allow for a BaseChatModel, I would also suggest changing the default prompt.

Currently, it lets the language model choose whether to use python code or not, which works with text-davinci-003, but leads to an invalid answer with gpt-3.5-turbo (something like "Sorry, as a Language Model I cannot...."), when running the example from the docs. I would modify it to ask the model only for python code. I cannot imagine any problem that a language model can confidently solve by itself that it cannot solve via python code.

[ghost]

1. Cannot initialize match chain with ChatOpenAI LLM

llm_math = LLMMathChain(llm=ChatOpenAI(temperature=0))

ValidationError Traceback (most recent call last) Cell In[33], line 1 ----> 1 llm_math = LLMMathChain(llm=ChatOpenAI(temperature=0))

File ~/anaconda3/envs/gpt_index/lib/python3.8/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.init()

ValidationError: 1 validation error for LLMMathChain llm Can't instantiate abstract class BaseLLM with abstract methods _agenerate, _generate, _llm_type (type=type_error)

2. Works ok with OpenAI LLM

llm_math = LLMMathChain(llm=OpenAI(temperature=0))

I am facing this issue with both SQLDatabaseChain + ChatOpenAI and SQLDatabaseChain + ChatOpenAI. What way is there around this?

[cnndabbler]

from langchain import LLMMathChain llm_math = LLMMathChain(llm=ChatOpenAI(temperature=0))

now generates no error :). Thank you for fixing this !

[mzhadigerov]

Are there any fixes for this? I'm unable to instantiate VectorStoreRouterToolkit with ChatOpenAI as an llm

[stakes]

i think i fixed it with https://github.com/hwchase17/langchain/pull/3808