Make agent handle normal conversation that are not covered by tools

[andyxinyuwu]

Hi, I am building a chatbot that could answer questions related to some internal data. I have defined an agent that has access to a few tools that could query our internal database. However, at the same time, I do want the chatbot to handle normal conversation well beyond the internal data.

For example, when the user says nice, the agent responds with Thank you! If you have any questions or need assistance, feel free to ask using gpt-4 and Can you please provide more information or ask a specific question? using gpt-3.5, which is not ideal.

I am not sure how langchain handle message like nice. If I directly send nice to chatGPT with gpt-3.5, it responds with Glad to hear that! Is there anything specific you'd like to ask or discuss?, which proves that gpt-3.5 has the capability to respond well. Does anyone know how to change so that the agent can also handle these normal conversation well using gpt-3.5?

Here are my setup

        self.tools = [
            Tool(
                name="FAQ",
                func=index.query,
                description="useful when query internal database"
            ),
        ]

        prompt = ZeroShotAgent.create_prompt(
            self.tools,
            prefix=prefix,
            suffix=suffix,
            format_instructions=FORMAT_INSTRUCTION,
            input_variables=["input", "chat_history-{}".format(user_id), "agent_scratchpad"]
        )

        llm_chain = LLMChain(llm=ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo"), prompt=prompt)
        agent = ZeroShotAgent(llm_chain=llm_chain, tools=self.tools)
        memory = ConversationBufferMemory(memory_key=str('chat_history'))
        agent_executor = AgentExecutor.from_agent_and_tools(
                    agent=self.agent, tools=self.tools, verbose=True,
                    max_iterations=2, early_stopping_method="generate", memory=memory)

[tbscode]

Think this is what the Conversation Agent is for.

I've tried it mostly identifies when not to use a tool and then normally continues the conversation.

[andyxinyuwu]

Thanks @tbscode. I have tried Conversation Agent as well. I find it sometimes return message that cannot be parsed by this function, meaning the response doesn't follow this format r"Action: (.*?)[\n]*Action Input: (.*)". This will cause the agent returning error. Also in a lot of times, it is not having a conversation either. For example, when user send nice, it generates

> Entering new AgentExecutor chain...
Thought: This is not a valid question.
Final Answer: I'm sorry, I don't understand the question. Can you please provide more information or ask a specific question

[exiao]

I ran into the exact same issues, and I had to overwrite the ZeroShotAgent’s parser (which has been moved into OutputParser, so this code is a bit outdated).

ZeroShotAgent._extract_tool_and_input = _extract_tool_and_input

Here’s the output parser I wrote for MRKL so it will always return something, even if it doesn’t find a match

def _extract_tool_and_input(self, text: str) -> Optional[Tuple[str, str]]:
    """Overwriting output parser temporarily"""
    if "Final Answer:" in text:
        return "Final Answer", text.split("Final Answer:")[-1].strip()
    regex = r"Action\s*\d*\s*:(.*?)\nAction\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)"
    
    match = re.search(regex, text, re.DOTALL)
    if not match:
        return "Final Answer", text # This is usually where an error is thrown
    action = match.group(1)
    action_input = match.group(2)
    return action.strip(), action_input.strip(" ").strip('"')

[ChanChar]

Throwing out another option that we were looking at: returnDirect: true can be set on any tool and the output of the tool will be returned immediately, short circuiting the agent loop/scratchpad.

It still requires to use the agent's output parser (or your own custom one) to get the results out of the agent's initial pass.

export class CustomTool extends Tool {
  name = 'custom-tool';
  description = 'used to handle custom things';
  returnDirect = true;

  async _call(input: string) {
    ...

    const agentParser = new ChatConversationalAgentOutputParser();
    const parsedResponse = await agentParser.parse(someProcess.text);

    return parsedResponse.returnValues?.output;
  }
}

[andyxinyuwu]

Thanks! I end up overriding _extract_tool_and_input too