langroid icon indicating copy to clipboard operation
langroid copied to clipboard

handle_llm_no_tool is ignored in TableChatAgent due to missing check in handle_message_fallback

Open parvk11 opened this issue 6 months ago • 3 comments

Describe the bug The handle_llm_no_tool setting in agent config is respected in the base ChatAgent via its handle_message_fallback method, but ignored in subclasses like TableChatAgent because they override the method without checking self.config.handle_llm_no_tool.

To Reproduce Steps to reproduce the behavior:

  1. Define a TableChatAgentConfig with handle_llm_no_tool="LLM forgot to use a tool."
  2. Create a TableChatAgent (or a custom agent that inherits from TableChatAgent) using this config
  3. Let the LLM emit a message without using a required tool (e.g. pandas_eval)
  4. The handle_llm_no_tool message is not returned, even though it is set

Expected behavior When handle_llm_no_tool is set in the config, agents should check for it and return the message when the LLM fails to use a required tool.

Screenshots

I have a data regression agent that is a subclass of the table chat agent, here is its response when the LLM doesn't call a tool - Image

The expected response that I set in the code is this - Image

Possible Solution

Update subclass override in TableChatAgent.handle_message_fallback to explicitly check for self.config.handle_llm_no_tool before executing custom logic, as implemented in ChatAgent.handle_message_fallback

I'm happy to submit a fix if that would be helpful.

parvk11 avatar Jun 22 '25 19:06 parvk11

Thanks for raising this ! I'll think of a good way to handle this.

pchalasani avatar Jun 23 '25 01:06 pchalasani

Actually, now that I'm thinking about this, when using an agent that has a specialized handle_message_fallback method explicitly defined, one should not be setting the handle_llm_no_tool parameter at all , because that fundamentally conflicts with having a custom handle_message_fallback method.

pchalasani avatar Jun 23 '25 20:06 pchalasani

Actually, now that I'm thinking about this, when using an agent that has a specialized handle_message_fallback method explicitly defined, one should not be setting the handle_llm_no_tool parameter at all , because that fundamentally conflicts with having a custom handle_message_fallback method.

Ok, thanks, that makes sense. One follow-up: if I’m building a custom agent that inherits from a specialized agent and I want to customize fallback behavior for this agent, should I then override handle_message_fallback again in my subclass, rather than using the handle_llm_no_tool parameter?

parvk11 avatar Jun 25 '25 00:06 parvk11