FileReadTool Failing Consistently

[kjenney]

I have the following script. I ran this a month ago and it ran fine. Now it's failing on FileReadTool.

import os, sys
from crewai import Agent, Task, Process, Crew
from langchain_community.llms import Ollama
from crewai_tools import FileReadTool

script_dir = os.path.dirname(__file__)
haiku_filename = "haiku.txt"
file_read_tool = FileReadTool(file_path=f"{script_dir}/haiku.txt")
llm_mixtral = Ollama(model="dolphin-mistral")

writer = Agent(
    role="Haiku Expert",
    goal="Write Haiku",
    backstory="You love Haiku",
    verbose=True,
    allow_delegation=False,
    llm=llm_mixtral
)

reviewer = Agent(
    role="Haiku Expert",
    goal="Review Haikus",
    backstory="You review Haikus",
    verbose=True,
    allow_delegation=False,
    llm=llm_mixtral
)

write = Task(
    description="Write a short haiku about the ocean.",
    agent=writer,
    expected_output="A string containing a short haiku about the ocean saved to a file.",
    output_file=haiku_filename
)

review = Task(
    description=f"Read a haiku from {haiku_filename} and let me know if it's actually a haiku.",
    agent=reviewer,
    expected_output=f"A string containing a review of {haiku_filename}.",
    tools=[file_read_tool]
)

crew = Crew(
    agents=[writer,reviewer],
    tasks=[write,review],
    verbose=2,
    process=Process.sequential
)

result = crew.kickoff()

print("######################")
print(result)

This is the output:

 [DEBUG]: == Working Agent: Haiku Expert
 [INFO]: == Starting Task: Write a short haiku about the ocean.


> Entering new CrewAgentExecutor chain...
 A glistening deep
Endless waves whisper secrets
Life's rhythm, ocean keeps.

Final Answer:
A glistening deep,
Endless waves whisper secrets,
Life's rhythm, ocean keeps.

> Finished chain.
 [DEBUG]: == [Haiku Expert] Task output: A glistening deep,
Endless waves whisper secrets,
Life's rhythm, ocean keeps.


 [DEBUG]: == Working Agent: Haiku Expert
 [INFO]: == Starting Task: Read a haiku from haiku.txt and let me know if it's actually a haiku.


> Entering new CrewAgentExecutor chain...
 Thought: To determine if a given text from haiku.txt is actually a haiku, I need to read its content first.

Action: Read a file's content
Action Input: {'file_name': '/Users/ken/devel/ai/agents/crewai-agents-save-a-haiku/haiku.txt'}

A glistening deep,
Endless waves whisper secrets,
Life's rhythm, ocean keeps.

 Thought: To assess whether the given text is a haiku or not, I will analyze its structure and content as per the classical haiku rules.

Action: Read a file's content
Action Input: {'file_name': '/Users/ken/devel/ai/agents/crewai-agents-save-a-haiku/haiku.txt'}

I tried reusing the same input, I must stop using this action input. I'll try something else instead.

The output keeps spitting out I tried reusing the same input... continuously. The documentation here mentions file types. If this is an issue with the format, it would be helpful to have some guidance on what the correct format should be.

[joaomdmoura]

This feels like a problem with the model, we have a caching layer that prevents the model from using the same tool consecutively with the same arguments, because it just did we try to prevent it from doing it again and instead move on with the task, but the model seem to keep trying to use it, have you tried different models to see if you get similar resutls?

[kjenney]

Yeah, it's the local model. Using Ollama causes it to get stuck in an infinite loop. Using OpenAI it works fine. I don't understand why the tool needs to care at all about which model is being used. This is not my first issue with trying to run Ollama with CrewAI.

[vmsaif]

it also happens with langchain_google_genai ->

GoogleGenerativeAI(model="gemini-pro", google_api_key=os.getenv("GOOGLE_API_KEY"), verbose = True, temperature = 0.2, )

    Any alternative solution using Gemini?

[GregHilston]

@kjenney or @vmsaif have you made any new discoveries on this topic? As I'm investigating it as well

[vmsaif]

@kjenney or @vmsaif have you made any new discoveries on this topic? As I'm investigating it as well

Yes, I haven't tested yet but I think this has to do something with if the model supports tool calling. I found these 2 links. These list what is supported or not. I will test them today.

https://python.langchain.com/docs/integrations/chat/

https://python.langchain.com/docs/integrations/llms/

Also let us know about any discoveries

[ViktorKarpilov]

I solved current issue by utilization of a specific "Exit" task where I explicitly specified that the only action required is to exit. This approach works well, even for models without tool support. However, if an error occurs, the issue is unfortunatly reproduced.

[kjenney]

I solved current issue by utilization of a specific "Exit" task where I explicitly specified that the only action required is to exit. This approach works well, even for models without tool support. However, if an error occurs, the issue is unfortunatly reproduced.

Do you have example code?

[kjenney]

I tried using openhermes with a low temp and also using an Exit task. Neither of these resulted in different results for me:

texit = Task(
    description="The only action required is to exit.",
    agent=reviewer,
    expected_output="Exit.",
)

[kjenney]

Switching to using ChatOpenAI instead of Ollama worked for me.

From: from langchain_community.llms import Ollama To: from langchain_openai import ChatOpenAI

Also, using OpenHermes with a low temp (custom Modelfile) worked, while using Dolphin Mistral (low temp and standard both) fails.

[NSMyself]

I'm running into this issue when running Groq's llama3-70b-8192 and when searching using Serper. It seems like the LLM does not particularly like the results, tries to search again with a different query but then it just stays there indefinitely.

This feels like a problem with the model, we have a caching layer that prevents the model from using the same tool consecutively with the same arguments, because it just did we try to prevent it from doing it again and instead move on with the task, but the model seem to keep trying to use it, have you tried different models to see if you get similar resutls?

@joaomdmoura I can see this happening over here:

(...)
Thought: I need to gather more specific information about XRP's roadmap to provide a clear and direct summary.

Action: Search the internet
Action Input: {"search_query": "Ripple's roadmap for 2023 and beyond"}

I tried reusing the same input, I must stop using this action input. I'll try something else instead.

Thought: I need to gather more specific information about XRP's roadmap to provide a clear and direct summary.

Action: Search the internet
Action Input: {"search_query": "Ripple's roadmap for 2023 and beyond"}

I tried reusing the same input, I must stop using this action input. I'll try something else instead.
(...)

Eventually it did manage to get out, although it took 130s when it usually takes 5s.

[mdkberry]

I am getting this too with local llms (mistral 7b with recommended modefile), annoyingly it resolves perfectly on the first go, but the agent thinks it hasnt finished and continues on trying different tasks rather than moving onto the next agent. In my case reading a local PDF file - which it does first go - and then thinking it hasnt, so tries various other things firstly using the following message then breaking with errors. so essentially it doesnt realise it has finished its task.

I tried reusing the same input, I must stop using this action input. I'll try something else instead.

[Amirjab21]

I keep getting this error:

File "/opt/anaconda3/envs/aigent/lib/python3.11/site-packages/pydantic/main.py", line 767, in getattr raise AttributeError(f'{type(self).name!r} object has no attribute {item!r}') AttributeError: 'FileReadTool' object has no attribute 'return_direct'

It might be that the files have more tokens than Llama3 can handle (from Ollama)

[Yembot31013]

I'm running into this issue when running Groq's llama3-70b-8192 and when searching using Serper. It seems like the LLM does not particularly like the results, tries to search again with a different query but then it just stays there indefinitely.

This feels like a problem with the model, we have a caching layer that prevents the model from using the same tool consecutively with the same arguments, because it just did we try to prevent it from doing it again and instead move on with the task, but the model seem to keep trying to use it, have you tried different models to see if you get similar resutls?

@joaomdmoura I can see this happening over here:

(...)
Thought: I need to gather more specific information about XRP's roadmap to provide a clear and direct summary.

Action: Search the internet
Action Input: {"search_query": "Ripple's roadmap for 2023 and beyond"}

I tried reusing the same input, I must stop using this action input. I'll try something else instead.

Thought: I need to gather more specific information about XRP's roadmap to provide a clear and direct summary.

Action: Search the internet
Action Input: {"search_query": "Ripple's roadmap for 2023 and beyond"}

I tried reusing the same input, I must stop using this action input. I'll try something else instead.
(...)

Eventually it did manage to get out, although it took 130s when it usually takes 5s.

Actually, you can also control that; you will let the model know if it should keep trying or not. You can do that in your prompting, which is going to be in the Task description and maybe expected_output too (if only description didn't fix the issue alone)

[Borillion]

I am getting this too with local llms (mistral 7b with recommended modefile), annoyingly it resolves perfectly on the first go, but the agent thinks it hasnt finished and continues on trying different tasks rather than moving onto the next agent. In my case reading a local PDF file - which it does first go - and then thinking it hasnt, so tries various other things firstly using the following message then breaking with errors. so essentially it doesnt realise it has finished its task.

I tried reusing the same input, I must stop using this action input. I'll try something else instead.

Whats the "recommended modefile" ?

[mdhuzaifapatel]

Yeah, it's the local model. Using Ollama causes it to get stuck in an infinite loop. Using OpenAI it works fine. I don't understand why the tool needs to care at all about which model is being used. This is not my first issue with trying to run Ollama with CrewAI.

I'm using Ollama - local model is Llama2-7b-chat I tried using FileReadTool() for reading a txt file, but I'm getting an error, the agent is not able to read the file. It's like this: Action 'Read a file's content(**kwargs: Any) -> Any' don't exist, these are the only available Actions: Tool Name: Read a file's content(**kwargs: Any) -> Any

I went to this method because, the output of Task1 was not being used by Task2, so I thought I will save output of Task1 in a txt file, then ask Task2 to read it and generate code for that data.

Both the methods are not working can you please help me out.

@joaomdmoura @mdkberry @Borillion @NSMyself @GregHilston

[GregHilston]

I was unable to figure this out and gave up on the problem and using crewai, due to how much time I spent on failing to figure this out unfortunately.

[vijayakrishna92]

anyone solved this issue? even i get this error whenever agent uses the tool( i am using mistral 7b instruct v0.3). for me it happens whenever page takes time to load and scraper unable to get the paragraph content.

[JavierCCC]

i'm in the same situation using llama3.1 instruct through nvdia NMI container...

[jirapongk]

i'm in the same situation using llama3.1 instruct through nvdia NMI container...

For Local Model like Ollama Could you please try adding something like "You have access to tools which can read files [file_read_tool(file_path: str)] and write files [file_write_tool(filename: str, content: str)]" to your task description? It might help LLMs easily infer function calls. Note: file_read_tool is the function name, and the text inside the parentheses is the function parameter description. You can change it to whatever function you want to use

[adsk2050]

This is happening because the last tool calls are being cached in ToolsHandler cache.

I am working on finding a way to reset its cache, or disable it by passing some argument when creating the Crew class. If anybody figures it out, kindly let me know