[fix] GPT sometimes ignores the SQL format request.

[skcoirz]

I found GPT3.5 sometimes consistently ignores the requested SQL format - “Action *** Action Input ***”, instead provided “Action ***, “”” when there is no Action Input. This will cause exception in the match check below.

Adding a secondary match to mitigate the problem as I couldn’t find a good way to force GPT to use the format all the time…

In the example below, you can find in my prints starting with “MK, “, the Action Input placeholder is missed in the beginning when query table name. This mitigation makes it pass and continue the flow.

[hwchase17]

do you have an example to easily reproduce this?

[skcoirz]

do you have an example to easily reproduce this?

This happens quite often in SQL agent. (the example in my screenshot is reproducible. Tested on Spark Agent.)

In other agents, (I tested with ZeroShot+DDG), I haven't observed so far. I think the issue could happen when the prompt (PREFIX+FORMAT_INSTRUCTIONS) is complex. GPT 3.5 becomes less focused on the format requirement instructions and shifts focus to other prompt words. The issue pointed in this PR is not the only way to fail. GPT can ignore the first key word “action” too when the chat_history is long or when the agent_scratchpad is long. Adding a fallback parser action could be a potential solution.

[baskaryan]

stale, output parser has since been meaningfully changed