SQL chain generates extra add on question if I use ChatOpenAI inplace of OpenAI

[charanhu]

System Info

import os from langchain import OpenAI, SQLDatabase, SQLDatabaseChain from langchain.chat_models import ChatOpenAI from dotenv import load_dotenv load_dotenv() os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") db = SQLDatabase.from_uri("sqlite:///data/data.db") llm = ChatOpenAI(temperature=0, verbose=True) db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True) res = db_chain.run("total sales by each region?") print(res)

But If I use text-davinici, It generates single result.

import os from langchain import OpenAI, SQLDatabase, SQLDatabaseChain from langchain.chat_models import ChatOpenAI from dotenv import load_dotenv load_dotenv() os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") db = SQLDatabase.from_uri("sqlite:///data/data.db") llm = OpenAI(temperature=0, verbose=True) db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True) res = db_chain.run("total sales by each region?") print(res)

how to overcome this issue in "ChatOpenAI"?

Who can help?

No response

Information

• [X] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [ ] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [ ] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [X] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

import os from langchain import OpenAI, SQLDatabase, SQLDatabaseChain from langchain.chat_models import ChatOpenAI from dotenv import load_dotenv load_dotenv() os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") db = SQLDatabase.from_uri("sqlite:///data/data.db") llm = ChatOpenAI(temperature=0, verbose=True) db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True) res = db_chain.run("total sales by each region?") print(res)

Expected behavior

I need one answer that is for user input query only. But after the answer It again adding a question by itself. Extra add on question and query not needed in ChatOpenAI

[huozhong-in]

I have the same issue. chain = load_qa_chain(llm=llm, chain_type='stuff') answer = chain.run(input_documents=docs, question=query, prompt=prompt, max_responses=1) print(answer) extra content/QA is not need!

[Qucy]

Same here, I use the sample code from https://python.langchain.com/en/latest/modules/agents/agents/examples/mrkl_chat.html and my current langchain version is 0.0.191

[oskrocha]

Same issue here. I think this issue is a duplicate of 4840

[levalencia]

Same here!

Good answer: There are 244 people in the Profiles table.

I didnt ask this! Question: What are the top 5 favorite sports? SQLQuery:SELECT TOP 5 [FavoriteSport], COUNT(*) as NumPeople FROM [Profiles] WHERE [FavoriteSport] IS NOT NULL GROUP BY [FavoriteSport] ORDER BY NumPeople DESC

[dosubot[bot]]

Hi, @charanhu! I'm Dosu, and I'm here to help the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue is that when using the ChatOpenAI model, it generates an extra add-on question after providing the answer to the user's query. Several users, including yourself, have reported experiencing this issue. One user suggested that this issue may be a duplicate of issue #4840.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the LangChain project!

[chriskuchar]

I am using langchain developmental, and I had to put the stop word 'What' as when it kept chaining it wouldn't do "Question:" as it did for @charanhu. For me, it would be "what is the ......?" as the continuation instead of "Question: what is the ...?" as the unwanted continuation. Adding '\nWhat' to the stop condition according to https://github.com/langchain-ai/langchain/issues/4840 did the trick for me.

[renlon]

Has this issue been fixed in https://github.com/langchain-ai/langchain/issues/4840. I am running to the same issue when using SageMaker endpoint. It seems that questions are concatenated along with the actual SQL query.

Question: How many log entries are there in the log file?

SQLQuery: SELECT COUNT(*) FROM alb_e29f4392d2b64810adcc20d3a1eae4db_1697521767;Failed to execute query.
Traceback (most recent call last):
  File "/opt/miniconda3/envs/minianda/lib/python3.8/site-packages/pyathena/common.py", line 520, in _execute
    query_id = retry_api_call(
  File "/opt/miniconda3/envs/minianda/lib/python3.8/site-packages/pyathena/util.py", line 68, in retry_api_call
    return retry(func, *args, **kwargs)
  File "/opt/miniconda3/envs/minianda/lib/python3.8/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
  File "/opt/miniconda3/envs/minianda/lib/python3.8/site-packages/tenacity/__init__.py", line 314, in iter
    return fut.result()
  File "/opt/miniconda3/envs/minianda/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/opt/miniconda3/envs/minianda/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/opt/miniconda3/envs/minianda/lib/python3.8/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
  File "/opt/miniconda3/envs/minianda/lib/python3.8/site-packages/botocore/client.py", line 535, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/opt/miniconda3/envs/minianda/lib/python3.8/site-packages/botocore/client.py", line 980, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidRequestException: An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: line 1:1: mismatched input 'Question'. Expecting: 'ALTER', 'ANALYZE', 'CALL', 'COMMENT', 'COMMIT', 'CREATE', 'DEALLOCATE', 'DELETE', 'DENY', 'DESC', 'DESCRIBE', 'DROP', 'EXECUTE', 'EXPLAIN', 'GRANT', 'INSERT', 'MERGE', 'PREPARE', 'REFRESH', 'RESET', 'REVOKE', 'ROLLBACK', 'SET', 'SHOW', 'START', 'TRUNCATE', 'UNLOAD', 'UPDATE', 'USE', <query>
2023-10-16 15:49:48,861 - MiniANDA - ERROR - Error in querying access log. Error: (pyathena.error.DatabaseError) An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: line 1:1: mismatched input 'Question'. Expecting: 'ALTER', 'ANALYZE', 'CALL', 'COMMENT', 'COMMIT', 'CREATE', 'DEALLOCATE', 'DELETE', 'DENY', 'DESC', 'DESCRIBE', 'DROP', 'EXECUTE', 'EXPLAIN', 'GRANT', 'INSERT', 'MERGE', 'PREPARE', 'REFRESH', 'RESET', 'REVOKE', 'ROLLBACK', 'SET', 'SHOW', 'START', 'TRUNCATE', 'UNLOAD', 'UPDATE', 'USE', <query>
[SQL: Question: How many log entries are there in the log file?

As you can see, since the question is included as part of the SQL query, the DB complains about the query syntax.