[Bug] Double quotation around SQL query issue in 06-langchain-agents.ipynb

[janzheng]

Is this a new bug?

• [X] I believe this is a new bug
• [X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When replicating in a Python notebook, the SQL Database tool produces this error, because of the double quotation marks ("") OperationalError: near ""SELECT stock_ticker, price, date FROM stocks WHERE (stock_ticker = 'ABC' OR stock_ticker = 'XYZ') AND (date = '2023-01-03' OR date = '2023-01-04') LIMIT 5"": syntax error

Expected Behavior

Single quotation marks: "SELECT stock_ticker, price, date FROM stocks WHERE (stock_ticker = 'ABC' OR stock_ticker = 'XYZ') AND (date = '2023-01-03' OR date = '2023-01-04') LIMIT 5"

This can be done by changing the description to:

sql_tool = Tool( ... ,description="Useful for when you need to answer questions about stocks and their prices. The SQL query should be outputted plainly, do not surround it in quotes or anything else.")

Steps To Reproduce

Run with colab

Relevant log output

No response

Environment

Colab and Replit

Additional Context

Adding the extra description seems to prevent double quotes

[JacobGoldenArt]

Yep. I'm running into this too. But even after updating the description suggested above, I'm getting the following error:

OperationalError                          Traceback (most recent call last)
[/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py](https://localhost:8080/#) in _execute_context(self, dialect, constructor, statement, parameters, execution_options, *args, **kw)
   1899                 if not evt_handled:
-> 1900                     self.dialect.do_execute(
   1901                         cursor, statement, parameters, context

22 frames
OperationalError: near ""SELECT (price/ (SELECT price FROM stocks WHERE stock_ticker = 'XYZ' AND date = '2023-01-03')) AS ratio_Jan3, (price/ (SELECT price FROM stocks WHERE stock_ticker = 'XYZ' AND date = '2023-01-04')) AS ratio_Jan4 FROM stocks WHERE stock_ticker = 'ABC' AND (date = '2023-01-03' OR date = '2023-01-04') LIMIT 5"": syntax error

[janzheng]

James left a comment in the video:

try preappending "Use sqlite syntax to answer this query:" to the prompt (thanks to @mrburns4031 and @memesofproduction27 for pointing this out)

https://www.youtube.com/watch?v=jSP-gSEyVeI&t=172s

[davidsilvasmith]

Thanks for the fix @janzheng. Had this issue on my mac terminal. Doing this as you said in your initial issue fixed it for me.

sql_tool = Tool( ... ,description="Useful for when you need to answer questions about stocks and their prices. The SQL query should be outputted plainly, do not surround it in quotes or anything else.")

Seems strange the description has anything to do with the output. Seems more like a command to an LLM.

[davidsilvasmith]

OK I see how the description works is explained in the next step.

print(zero_shot_agent.agent.llm_chain.prompt.template)

[davidsilvasmith]

The quotes command helped for the first example, but then on the next example failed with an issue. Using the prepending like this worked for both.

sql_tool = Tool(
    name='Stock DB',
    func=sql_chain.run,
    description="Use sqlite syntax to answer this query: Useful for when you need to answer questions about stocks " \
                "and their prices."
    
)

[jellederijke]

I still get similar syntax errors despite using different variations like:

from langchain.agents import Tool

sql_tool = Tool(
    name='Stock DB',
    func=sql_chain.run,
    description="Use sqlite syntax to answer this query: Useful for when you need to answer questions about stocks and their prices. The SQL query should be outputted plainly, do not surround it in quotes or anything else."
)

For instance this:

result = count_tokens( zero_shot_agent, "What is in the first row of my stocks database?" )

At first seems to create a nice query:

Entering new SQLDatabaseChain chain... SELECT * FROM stocks LIMIT 1;

But one line later messes it up again: SQLQuery: "SELECT * FROM stocks LIMIT 1;"

Which gives the OperationalError.

Any thought are very much appreciated :)