Strange error on processing response?

[silarsis]

I'm getting an error I think on trying to read the response from the llm. I'm using OpenAI via self.guide = guidance.llms.OpenAI(model='gpt-3.5-turbo'), and certain questions trigger the traceback below - not all questions, it seems to be fairly arbitrary. Looking at the code, it possibly needs a guard against empty string responses?

agent      |   File "/usr/local/lib/python3.9/site-packages/guidance/library/_gen.py", line 133, in gen
agent      |     gen_obj = await parser.llm_session(
agent      |   File "/usr/local/lib/python3.9/site-packages/guidance/llms/_openai.py", line 310, in __call__
agent      |     out = self.llm.caller(**call_args)
agent      |   File "/usr/local/lib/python3.9/site-packages/guidance/llms/_openai.py", line 192, in _library_call    
agent      |     out = add_text_to_chat_mode(out)
agent      |   File "/usr/local/lib/python3.9/site-packages/guidance/llms/_openai.py", line 57, in add_text_to_chat_mode
agent      |     c['text'] = c['message']['content']
agent      |   File "/usr/local/lib/python3.9/site-packages/openai/openai_object.py", line 71, in __setitem__        
agent      |     raise ValueError(
agent      | ValueError: You cannot set text to an empty string. We interpret empty strings as None in requests.You may set {
agent      |   "finish_reason": "stop",
agent      |   "index": 0,
agent      |   "message": {
agent      |     "content": "",
agent      |     "role": "assistant"
agent      |   }
agent      | }.text = None to delete the property

[silarsis]

if it helps, my prompt is:

self._prompt_template = guidance("""
{{#system~}}
{{character}}

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{{tool_names}}]
Action Input: the input to the action

History of chat so far:
{{await 'history'}}
{{~/system}}
{{#user~}}
Question: {{await 'query'}}
{{~/user}}
{{#assistant~}}
{{gen 'answer' max_tokens=100 stop='Question:'}}
{{~/assistant}}
""", llm=self.guide, character=character, tool_names=[tool.name for tool in self.tools])

Then called with response = self._prompt_template(query=query, history=history)

[slundberg]

Thanks for sharing! I can't seem to reproduce the error with the following. What should I change?

import guidance

llm = guidance.llms.OpenAI(model='gpt-3.5-turbo')

character = "joe"
tool_names = ["saw", "hammer", "screwdriver"]

program = guidance("""
{{#system~}}
{{character}}

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{{tool_names}}]
Action Input: the input to the action

History of chat so far:
{{await 'history'}}
{{~/system}}
{{#user~}}
Question: {{await 'query'}}
{{~/user}}
{{#assistant~}}
{{gen 'answer' max_tokens=100 stop='Question:'}}
{{~/assistant}}
""", llm=llm, character=character, tool_names=tool_names)

program(query="how can I climb a tree?", history="")

[silarsis]

Not sure - I've moved on to a non-chat model (davinci instead of gpt for the model name) and am no longer getting it. But it happened on some prompts, not others - questions like "what are your tools?" seemed to trigger it, but a rewording of that question did not.

[slundberg]

Got it. I'll reopen this if you run across it again!

[nc]

I can repro this with the following code:

import datetime
import re
from typing import Any, Dict, List

guidance.llm = guidance.llms.OpenAI("gpt-4-32k")

tools: List[Dict[str, Any]] = [{
    'name': 'time',
    'description': 'A clock that tells the current time',
    'fn': lambda *args: datetime.datetime.now().strftime("%H:%M:%S")
},{
    'name': 'date',
    'description': 'A clock that tells the current date',
    'fn': lambda *args: datetime.datetime.now().strftime("%m/%d/%Y")
}]

action_pattern = r'Action: (.*?)'
action_input_pattern = r'Action Input: (.*?)'
final_answer_pattern = r'Final Answer: (.*?)'

cot = guidance('''
{{#system~}}
You are a helpful assistant.

Answer the following questions as best you can. You have access to the following tools: 
{{#each tools}}
{{this.name}} ({{this.description}})
{{~/each}}

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{{~#each tools}}{{this.name}}, {{~/each}}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!
Queetion: {{question}}
{{~/system}}
{{~#geneach 'conversation' stop=False}}
{{#assistant~}}
{{gen 'this.thought' temperature=0 max_tokens=1500 stop="Action:"}}
{{~/assistant}}
{{#assistant~}}
{{gen 'this.action' temperature=0 max_tokens=1500 stop="Action Input:"}}
{{~/assistant}}
{{#assistant~}}
{{gen 'this.action_input' temperature=0 max_tokens=1500 stop="Observation:"}}
{{~/assistant}}
{{#user~}}
<observation>{{set this.observation (await 'observation')}}</observation>
{{~/user}}
{{~/geneach}}''')
            
question = 'What is the date and time?'

out: guidance.Program = cot(question=question, tools=tools)

print(out)

for i in range(len(out['conversation'])):
    print("Conversation", i)
    conversation = out['conversation'][i]
    thought = conversation['thought']
    action_match = re.search(action_pattern, conversation['action'])
    action_input_match = re.search(action_input_pattern, conversation['action_input'])
    if 'Final Answer:' in conversation:
        final_answer_match = re.search(final_answer_pattern, conversation['final_answer'])
        if final_answer_match is None:
            print("Final answer not found")
            exit(1)
        else:
            print("Final Answer:", final_answer_match.group(1))
            exit(0)

    elif action_match is not None and action_input_match is not None:
        action = action_match.group(1)
        action_input = action_input_match.group(1)
        print("Action:", action)
        print("Action Input:", action_input)
        tool = next((tool for tool in tools if tool['name'] == action), None)
        if tool:
            print("Tool:", tool['name'])
            observation = tool['fn'](action_input) or "None"
            print("Observation:", observation)
            print(out(observation=observation))
        else: 
            print("Tool not found:", action)

    else:
        print("Action or Action Input not found")

[yueqing-wolai]

got same error, any solutions?

[MarkPopovich]

pip install --upgrade guidance