Weird behavior on how the parameters are passed or processed by an Agent that uses a custom Tool

[ing-norante]

I have defined a custom tool that fetches a local API as this search_api_tool.py:

import json
from langchain.tools import tool
from dotenv import load_dotenv
import requests
import os

load_dotenv()  # This loads the environment variables from .env

@tool
def search_api_tool(input_string: str) -> dict:
    """
    Tool to fetch conversation counts from the Search API.
    Expects 'input_string' as a JSON string containing 'start_date', 'end_date', and 'symbol'.
    """
    try:
        params = json.loads(input_string)
    except json.JSONDecodeError:
        return {"error": "Invalid input format. Expected JSON string."}


    # Check for either 'keyword' or 'symbol' and use whichever is available
    symbol = params.get("symbol") or params.get("keyword")
    start_date = params.get("start_date")
    end_date = params.get("end_date")

    if not all([symbol, start_date, end_date]):
        return {"error": "Missing required parameters."}

    url = f"http://localhost:3000/api/messages?start_date={start_date}&end_date={end_date}&symbol={symbol}"
    headers = {
        'Authorization': f'Bearer {os.getenv("API_KEY")}'
    }

    try:
        response = requests.get(url, headers=headers)
        response.raise_for_status()
        return response.json()
    except requests.RequestException as e:
        return {"error": str(e)}

In my main.py I've defined my custom tool for the agent as:

# Create a custom tool for the agent
custom_tool = Tool(
    name="SearchAPI",
    func=search_api_tool,  # Directly use the search_api_tool
    description="Tool to fetch conversation counts from the Search API."
)

# Create an agent
market_analyst = Agent(
  role='Market Analyst',
  goal='Analyze market trends based on conversation data.',
  backstory='A skilled analyst with expertise in identifying market trends from conversation data.',
  tools=[custom_tool],
  verbose=True
)


# Function to prompt user for input
def prompt_user_for_input():
    symbol = input("What is the symbol you want to analyze? ")
    start_date = input("What is the start date? (YYYY-MM-DD) ")
    end_date = input("What is the end date? (YYYY-MM-DD) ")
    return symbol, start_date, end_date

# Get user input
symbol, start_date, end_date = prompt_user_for_input()

# Custom method for executing the task with the correct parameters
def execute_market_analysis_task(agent, tool, symbol, start_date, end_date):
    params_str = json.dumps({"symbol": symbol, "start_date": start_date, "end_date": end_date})
    return tool.run(params_str)

# Define a task for the agent with user-provided parameters
market_analysis_task = Task(
    description=f'Analyze market conversations for {symbol} between {start_date} and {end_date}.',
    agent=market_analyst,
    tools=[custom_tool],
    execute=execute_market_analysis_task(market_analyst, custom_tool, symbol, start_date, end_date)
)

I've prepared a test.py file to debug and when passing the correct JSON as input the search_api_tool function outputs the correct stuff:

import json
from search_api_tool import search_api_tool
# Test the search_api_tool directly
test_input = json.dumps({"start_date": "02/02/2021", "end_date": "15/02/2021", "symbol": "AAPL"})
test_output = search_api_tool(test_input)
print("Test output:", test_output)

outputs:

python test.py
Test output: {'2021-02-02': 15, '2021-02-03': 43, '2021-02-04': 80, '2021-02-05': 49, '2021-02-06': 35, '2021-02-07': 36, '2021-02-08': 34, '2021-02-09': 49, '2021-02-10': 43, '2021-02-11': 46, '2021-02-12': 49, '2021-02-13': 43, '2021-02-14': 28}

But when I try to execute my main.py file, on the first call, it always mismatches the parameters and does weird stuff as replacing symbol with keyword. The issue here seems to be with how the input parameters are being formatted and passed to the search_api_tool. Initially, the agent is passing the parameters as a comma-separated list (AAPL, 2020-05-19, 2020-10-18), which leads to an error since the tool expects a JSON string. The agent then self-corrects by formatting the input as a JSON string, but the keys used ("keyword", "start_date", "end_date") might not match what the search_api_tool is expecting.

What am I missing here? I just want the Agent to call the API with the simple JSON as in the test file... Thank you very much for any suggestions!

[svickers]

In your docstring for search_api_tool be specific and give a concrete example of what you expect.

:param str input_string: a JSON serialized object containing the properties: start_date, end_date, symbol
Example:
{
    "start_date":"2023-04-20",
    "end_date":"2023-04-21",
    "symbol":"AAPL"
}

[joaomdmoura]

@svickers is a good suggestions, there is an option of creating multiple input tools as well, maybe we are missing documentation around that.

@ing-norante let us know how that doinf

[ing-norante]

Hello, and thanks for your suggestions! I've tried modifying the docstring for search_api_tool as this:

@tool
def search_api_tool(input_string: str) -> dict:
    """
    Tool to fetch conversation counts from the Search API.
    :param str input_string: a JSON serialized object containing the properties: start_date, end_date, symbol
    Example:
    {
        "start_date":"2023-04-20",
        "end_date":"2023-04-21",
        "symbol":"AAVE"
    }
    """

But when I try to execute the main script I'm always getting that strange format, see below:

> Entering new CrewAgentExecutor chain...
Thought: Do I need to use a tool? Yes
Action: SearchAPI
Action Input: AAPL, 2020-07-14, 2020-08-26{'error': 'Invalid input format. Expected JSON string.'}I need to correct the format of my input to match what the SearchAPI expects, which is a JSON string.
Action: SearchAPI
Action Input: {"term": "AAPL", "start_date": "2020-07-14", "end_date": "2020-08-26"}{'error': 'Missing required parameters.'}I seem to be missing some required parameters in my input. I need to check which parameters are required and make sure I include them in my query.

I'm not getting why it's using "term" instead of "symbol"... any help would be very appreciated!