[BUG] Tool inputs being fed through as an array instead of dictionary

[adilazmoon]

Description

Peculiar bug which started occurring when updating from 0.51.0 to 0.63.1. Tools are occasionally being fed arrays of dictionaries instead of dictionary objects. Seems to happen when the agent is expecting to run the tool multiple times for different parts of the task. Double checked, doesn't happen on 0.51.0

Steps to Reproduce

1. Give the agent a task to read 2 files, reddit_ouput.json and news_output.json
2. it reads the first file correctly, then attempts the following input on subsequent tool run: ` Agent: Data Analyst

Using tool: Read a file's content Tool Input: "{"file_path": "reddit_output.json"}" Tool Output: { "articles": [ { "title": "Pokémon Scarlet and Violet DLC The Hidden Treasure of Area Zero Announced", "url": "https://www.polygon.com/2023/10/15/pokemon-scarlet-violet-dlc-trailer-release-date", "snippet": "Pokémon Scarlet and Violet DLC The Hidden Treasure of Area Zero comes with a new story..." }, { "title": "Pokémon GO: Return of the Halloween Event 2023", "url": "https://www.gamerant.com/pokemon-go-halloween-event-2023/", "snippet": "The Halloween event in Pokémon GO returns with special bonuses and challenges..." }, { "title": "The Pokémon Company Teases New Game Announcements", "url": "https://www.pcgamer.com/pokemon-company-new-game-announcements/", "snippet": "The Pokémon Company has hinted at new upcoming game announcements and updates..." }, { "title": "Pokémon TCG: Sword and Shield Series Coming to an End", "url": "https://www.dexerto.com/pokemon/pokemon-tcg-sword-and-shield-series-coming-to-an-end-2023-2038614/", "snippet": "The Sword and Shield series of Pokémon TCG will conclude by the end of 2023..." }, { "title": "Pokémon Trading Card Game Classic Announced", "url": "https://www.ign.com/articles/pokemon-card-game-classic-announced", "snippet": "The Pokémon Company has officially announced the Pokémon Trading Card Game Classic..." }, { "title": "How to Catch Legendary Pokémon in Pokémon GO", "url": "https://www.gamepur.com/guides/how-to-catch-legendary-pokemon-in-pokemon-go", "snippet": "A guide on catching legendary Pokémon in Pokémon GO..." }, { "title": "The Rise of Pokémon Games on Mobile", "url": "https://www.techradar.com/news/the-rise-of-pokemon-games-on-mobile", "snippet": "An in-depth look at the growing trend of Pokémon games on mobile devices..." }, { "title": "Pokémon: The 10 Most Powerful Pokémon in the TCG", "url": "https://www.sbs.com.au/guide/article/pokemon-the-10-most-powerful-pokemon-in-the-tcg/owsegda1", "snippet": "Ranking the ten most powerful Pokémon in the Pokémon Trading Card Game..." }, { "title": "Pokémon Sleep: Tips and Tricks to Get the Most Out of Your Experience", "url": "https://www.polygon.com/2023/10/14/pokemon-sleep-tips-tricks", "snippet": "Tips and tricks for maximizing experience in Pokémon Sleep..." }, { "title": "Pokémon Legends Arceus: Tips to Level Up Fast", "url": "https://www.techtimes.com/articles/295677/20231014/pokemon-legends-arceus-tips-to-level-up-fast.htm", "snippet": "Strategies to level up quickly in Pokémon Legends Arceus..." } ] }

Agent: Data Analyst

Using tool: Read a file's content Tool Input: "[{"file_path": "news_output.json"}, {"articles": [{"title": "Pok\u00e9mon Trading Card Game Pocket Gets New Gameplay Trailer, Out October", "url": "https://www.nintendolife.com/news/2024/09/video-pokemon-trading-card-game-pocket-gets-new-gameplay-trailer-out-october", "date": "Fri 20th Sep 2024", "source": "Nintendo Life", "snippet": "Following a release date announcement at the World Championships in August, The Pok\u00e9mon Company has now released a gameplay video for its upcoming mobile title Pok\u00e9mon Trading Card Game Pocket..."}, {"title": "Pok\u00e9mon GO Confirms Debut of New Dynamax Pokemon", "url": "https://gamerant.com/pokemon-go-dynamax-falinks-scorbunny-sobble-grookey-debut/", "date": "3 hours ago", "source": "Game Rant", "snippet": "Pokemon GO announces the addition of new Dynamax Pokemon, and they will be available for trainers to capture within the next few days."}, {"title": "Pokemon GO Bug Makes It Impossible To Catch One of The Rarest Pokemon", "url": "https://gamerant.com/pokemon-go-phd-pikachu-impossible-catch-bug/", "date": "1 day ago", "source": "Game Rant", "snippet": "A Pokemon GO bug prevents users from catching one of the rarest Pokemon in the game and fans are hoping that it will be addressed in the next update."}, {"title": "Nintendo sues \u2018Pokemon with guns\u2019 game Palworld", "url": "https://www.independent.co.uk/tech/nintendo-pokemon-palworld-lawsuit-guns-b2615586.html", "date": "5 days ago", "source": "Independent", "snippet": "The Japanese company alleges that the Palworld video game 'infringes multiple patent rights'."}, {"title": "New Pok\u00e9mon Animated Short Is Seriously Adorable", "url": "https://www.nintendolife.com/news/2024/09/random-new-pokemon-animated-short-is-seriously-adorable", "date": "Tue 17th Sep 2024", "source": "Nintendo Life", "snippet": "There\u2019s no shortage of adorable animated Pok\u00e9mon content out there, but The Pok\u00e9mon Company\u2019s latest effort, \u2018Chasing the Moon,\u2019 is one of the best-looking ones we\u2019ve seen in a while."}]}]" Tool Output: Error: the Action Input is not a valid key, value dictionary.` Seems to consistently happen on tasks which require a tool to be called multiple times.

Expected behavior

Previously it could intelligently run the tool multiple separate times (in sequence).

Screenshots/Code snippets

Agent: Data Analyst

Task: 1. URL Processor and Web Scraper

Reads the JSON file reddit_output.json and news_output.json. Extracts URLs for each article

Agent: Data Analyst

Using tool: Read a file's content

Tool Input:

"{"file_path": "reddit_output.json"}"

Tool Output:

{ "articles": [ { "title": "Pokémon Scarlet and Violet DLC The Hidden Treasure of Area Zero Announced", "url": "https://www.polygon.com/2023/10/15/pokemon-scarlet-violet-dlc-trailer-release-date", "snippet": "Pokémon Scarlet and Violet DLC The Hidden Treasure of Area Zero comes with a new story..." }, { "title": "Pokémon GO: Return of the Halloween Event 2023", "url": "https://www.gamerant.com/pokemon-go-halloween-event-2023/", "snippet": "The Halloween event in Pokémon GO returns with special bonuses and challenges..." }, { "title": "The Pokémon Company Teases New Game Announcements", "url": "https://www.pcgamer.com/pokemon-company-new-game-announcements/", "snippet": "The Pokémon Company has hinted at new upcoming game announcements and updates..." }, { "title": "Pokémon TCG: Sword and Shield Series Coming to an End", "url": "https://www.dexerto.com/pokemon/pokemon-tcg-sword-and-shield-series-coming-to-an-end-2023-2038614/", "snippet": "The Sword and Shield series of Pokémon TCG will conclude by the end of 2023..." }, { "title": "Pokémon Trading Card Game Classic Announced", "url": "https://www.ign.com/articles/pokemon-card-game-classic-announced", "snippet": "The Pokémon Company has officially announced the Pokémon Trading Card Game Classic..." }, { "title": "How to Catch Legendary Pokémon in Pokémon GO", "url": "https://www.gamepur.com/guides/how-to-catch-legendary-pokemon-in-pokemon-go", "snippet": "A guide on catching legendary Pokémon in Pokémon GO..." }, { "title": "The Rise of Pokémon Games on Mobile", "url": "https://www.techradar.com/news/the-rise-of-pokemon-games-on-mobile", "snippet": "An in-depth look at the growing trend of Pokémon games on mobile devices..." }, { "title": "Pokémon: The 10 Most Powerful Pokémon in the TCG", "url": "https://www.sbs.com.au/guide/article/pokemon-the-10-most-powerful-pokemon-in-the-tcg/owsegda1", "snippet": "Ranking the ten most powerful Pokémon in the Pokémon Trading Card Game..." }, { "title": "Pokémon Sleep: Tips and Tricks to Get the Most Out of Your Experience", "url": "https://www.polygon.com/2023/10/14/pokemon-sleep-tips-tricks", "snippet": "Tips and tricks for maximizing experience in Pokémon Sleep..." }, { "title": "Pokémon Legends Arceus: Tips to Level Up Fast", "url": "https://www.techtimes.com/articles/295677/20231014/pokemon-legends-arceus-tips-to-level-up-fast.htm", "snippet": "Strategies to level up quickly in Pokémon Legends Arceus..." } ] }

Agent: Data Analyst

Using tool: Read a file's content

Tool Input:

"[{"file_path": "news_output.json"}, {"articles": [{"title": "Pok\u00e9mon Trading Card Game Pocket Gets New Gameplay Trailer, Out October", "url": "https://www.nintendolife.com/news/2024/09/video-pokemon-trading-card-game-pocket-gets-new-gameplay-trailer-out-october", "date": "Fri 20th Sep 2024", "source": "Nintendo Life", "snippet": "Following a release date announcement at the World Championships in August, The Pok\u00e9mon Company has now released a gameplay video for its upcoming mobile title Pok\u00e9mon Trading Card Game Pocket..."}, {"title": "Pok\u00e9mon GO Confirms Debut of New Dynamax Pokemon", "url": "https://gamerant.com/pokemon-go-dynamax-falinks-scorbunny-sobble-grookey-debut/", "date": "3 hours ago", "source": "Game Rant", "snippet": "Pokemon GO announces the addition of new Dynamax Pokemon, and they will be available for trainers to capture within the next few days."}, {"title": "Pokemon GO Bug Makes It Impossible To Catch One of The Rarest Pokemon", "url": "https://gamerant.com/pokemon-go-phd-pikachu-impossible-catch-bug/", "date": "1 day ago", "source": "Game Rant", "snippet": "A Pokemon GO bug prevents users from catching one of the rarest Pokemon in the game and fans are hoping that it will be addressed in the next update."}, {"title": "Nintendo sues \u2018Pokemon with guns\u2019 game Palworld", "url": "https://www.independent.co.uk/tech/nintendo-pokemon-palworld-lawsuit-guns-b2615586.html", "date": "5 days ago", "source": "Independent", "snippet": "The Japanese company alleges that the Palworld video game 'infringes multiple patent rights'."}, {"title": "New Pok\u00e9mon Animated Short Is Seriously Adorable", "url": "https://www.nintendolife.com/news/2024/09/random-new-pokemon-animated-short-is-seriously-adorable", "date": "Tue 17th Sep 2024", "source": "Nintendo Life", "snippet": "There\u2019s no shortage of adorable animated Pok\u00e9mon content out there, but The Pok\u00e9mon Company\u2019s latest effort, \u2018Chasing the Moon,\u2019 is one of the best-looking ones we\u2019ve seen in a while."}]}]"

Tool Output:

Error: the Action Input is not a valid key, value dictionary.

Operating System

Windows 11

Python Version

3.12

crewAI Version

0.63.1

crewAI Tools Version

0.12.1

Virtual Environment

Poetry

Evidence

Given above

Possible Solution

Was there a change in how the tools are called in recent updates? I see some changes in tool_usage.py, but nothing I can see would affect it this way.

Additional context

NA

[adilazmoon]

Seems to consistently occur when a tool is executed multiple times in a row. It looks like it appends the previous action output to an array containing the next action input and feeds that in as an input for the next execution

[joaomdmoura]

Looking into it @adilazmoon what specific model are you using? I think this could be the new check for funciton calling support

[joaomdmoura]

Was able to replicate the error, looking into it

[joaomdmoura]

seems to be adhoc, I got the issue once and not again, so probably the model getting confused, I'll test further just in case

[adilazmoon]

@joaomdmoura using OpenAI gpt-4o-mini. Yea it's also intermittent on my end, but definitely noticeable enough that it affects my output quite significantly and consistently.

[adilazmoon]

Okay the more correct thing for me to say is: I don't actually declare any model. Which I assume defaults to OpenAI gpt-4o-mini

[joaomdmoura]

I have an idea to make it better, will try it out. And yes the default is GPT-4o-mini 😊

[ANPCI]

@joaomdmoura I see the issue with Google Gemini-pro , just wanted to know if there is any update on the fix ?

Also, yes this is happening when a tool is executed multiple times in a row,

[ANPCI]

Okay the more correct thing for me to say is: I don't actually declare any model. Which I assume defaults to OpenAI gpt-4o-mini

Hi @adilazmoon , is it working for you now ?

[adilazmoon]

@ANPCI To be perfectly candid, I was sticking to 0.53.1 until this issue was resolved (ran some tests on the early 60 versions, but was still getting the issue). Decided to try and do a few test runs on the latest version 0.67.1 because you tagged me here. It actually looks like it's not an issue anymore. I'll do a few more runs just to be sure.

@joaomdmoura was there anything explicitly done on your side to fix this? Just curious about what the root cause was

[luisandino]

Hi, I am getting the same error. I made two versions of the same function, one receiving four parameters while the second is a dictionary, but I still get the error. The function worked well with the dictionary with an older library version, but now I am using v0.74.2, and it is failing. Are there any instructions on how to overcome this error message?

[sorin-costea]

Possibly the same issue as reported in https://github.com/crewAIInc/crewAI/issues/1522

[nate-walter]

I'm still seeing this bug today

[luisandino]

Hi, I am getting the same error. I made two versions of the same function, one receiving four parameters while the second is a dictionary, but I still get the error. The function worked well with the dictionary with an older library version, but now I am using v0.74.2, and it is failing. Are there any instructions on how to overcome this error message?

I have installed version 0.76.9, and it is working better now. The manager agent correctly calls the corresponding agent, task, and tool. However, sometimes, the agent controlling the tool responds by itself using the pre-trained data from the llm and won't call the tool, which is wrong. I want the agent to always call the tool. I will keep testing and post another update afterwards.

[sorin-costea]

@luisandino happens to me too, more as I increase the temperature. Setting it to 0 it works fine.

[luisandino]

@luisandino happens to me too, more as I increase the temperature. Setting it to 0 it works fine.

@sorin-costea Yes, it worked better by reducing the temperature to zero, still got a different response once after eight tests. I need to keep testing with different parameters and templates for the agent because I must ensure it calls the tool and not trying to infer the answer by itself.

We have to be very cautious with library updates, it can mess up what was working well before.

[hboregio]

Also happening with Composio's official crewai & google calendar example.

Got it to work by changing llm settings to: ChatOpenAI(model="gpt-4o-mini", temperature=0.7)

[jamesdhope]

Same with meta/llama3-70b when tool is called once or several times. CrewAI==0.80.0.

Tool Output:

Error: the Action Input is not a valid key, value dictionary.

[paarttipaabhalaji]

Same with meta/llama3-70b when tool is called once or several times. CrewAI==0.80.0.

Tool Output:

Error: the Action Input is not a valid key, value dictionary.

same for me with watsonx/mistralai/mistral-large model, crewAI==0.80.0

please resolve it quickly.

[alanjones2]

I found the same intermittent problem in v 0.86. It seems that one solution is to reduce the length/complexity of the task descriptions by splitting them into smaller tasks. That worked for me.

[SaurabhTayde]

I was also facing "Error: the Action Input is not a valid key, value dictionary."

llm: gemini-1.5-flash-001 / gemini-1.5-pro crewai==0.86.0 temperature=0.05

It was resolved when I set temperature to 0. Not sure how this bug is related to temp. But setting 0 solves it. Thanks @sorin-costea , @luisandino ! I hope in next version, crew doesn't throw this error with temp>0. And yes it is related to #1522

@bhancockio Is it possible for you to check this ?

FYI: @madhulokanath

[alanjones2]

In my case, setting the temperature to 0 made things worse.

What seems to be happening is that the LLM sometimes gives an unexpected response. So CrewAI says "ooops, better try that again".

As it is an LLM the responses vary. I've seen CrewAI loop around retrying multiple times until it gets a good response.

Setting the temperature to zero reduces the variation in the results, but it could go either way, either you get a good result every time, or, like me, you consistently get a bad one.

Setting the temp to zero gave me an infinite loop of "oops... try again" messages.

(Raising the temperature might have given a better result as I'd get more varied responses and so a higher probability of getting a good one!)

What worked was splitting up tasks into smaller sub-tasks. Which makes sense.

The more complex (or ambiguous or badly constructed) a task is, the more likely it is that the LLM gets confused and gives an unexpected result. Conversely, simplifying the tasks reduces that likelihood.

[SaurabhTayde]

@alanjones2 yeah, I was already creating small tasks only or breaking the complex one into multiple. Still had to face error. And, as we are working with not just LLM but hierarchial crew agents, so we should be creating easy to intermediate level tasks only, not the complex ones. Not sure till when this temperature workaround will work. Hope this gets resolved soon.

[alanjones2]

@SaurabhTayde Agreed, but it seems to have gone quiet.

[ShuminFu]

One of my observations is that when the input arguments of the Tool include nested json structures as well as boolean values, the formatter for the answer generated by LLM is more likely to format those booleans to lowercase "true"/"false". This causes an Error: the Action Input is not a valid key, value dictionary when using ast.literal_eval(), because Python expects capitalized boolean values (True/False).

My workaround is to tell the LLM to always generate answers with "True/False" in double quotation marks so that they can be pretended as strings and avoid being formatted.

[jtknox]

Hi @joaomdmoura,

I came across this thread due to the issue described by @ShuminFu, and I believe I've identified the culprit in the tool_usage.py file, specifically in the _original_tool_calling() method.

The issue stems from the use of ast.literal_eval() to evaluate the input as ShuminFu mentioned. While it ensures the input is treated as a literal Python dictionary, it runs into trouble with JSON-formatted inputs where boolean values differ in capitalization (true vs True). For example: {"booleanInputVar": true} Although this is valid JSON, passing it to ast.literal_eval() raises an error due to the lowercase true.

This problem seems specific to boolean inputs. A workaround is to declare all input schema fields as strings and then deserialize back into booleans within the tool itself. However, I suspect there might be a cleaner solution.

One approach could be to wrap ast.literal_eval() in a try-except block and attempt to parse the input with json.loads() if literal_eval fails. If json.loads() also fails, then the input would indeed be invalid.

Just wanted to add visibility to this and echo @ShuminFu's findings—ast.literal_eval() appears to be the root cause here.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[luisandino]

Do not stale

[niltonvasques]

Same happening here, with many different models.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.