How to avoid reaching maximum context length with repetitive task

[profnimrod]

I am using Crew AI to draft a short booklet on various subjects. The first task generates a list of essay titles (or book layout) given the title entered by the user. The second task then goes through each essay title (book section) and writes a 750-word essay for each. About halfway through I'm getting the error below. I think this is happening because the task that is writing the next essay for each section is including all the previous essays in the context when writing the next one. Assuming I have interpreted this correctly, how might I limit the context for this task?

`Exception has occurred: BadRequestError Error code: 400 - {'error': {'message': "This model's maximum context length is 8192 tokens. However, your messages resulted in 8327 tokens. Please reduce the length of the messages.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}} httpx.HTTPStatusError: Client error '400 Bad Request' for url 'https://api.openai.com/v1/chat/completions' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

During handling of the above exception, another exception occurred:

File "E:\EmergentPublications\WellnessPress\CrewAI\main.py", line 71, in result = crew.kickoff() ^^^^^^^^^^^^^^ openai.BadRequestError: Error code: 400 - {'error': {'message': "This model's maximum context length is 8192 tokens. However, your messages resulted in 8327 tokens. Please reduce the length of the messages.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}`

[Prayforhanluo]

some problem

[gururise]

Would be great if CrewAI could integrate MemGPT or some other type of method to manage context window without exceeding maximum.

[joaomdmoura]

There should be a guardrails for that already so I'll look into it and why it's not working. Some version of MemGPT integration is coming btw 😎👉👉

[gururise]

I have found the Nous Hermes2 Yi works very well, but is unusable for most tasks because it constantly exceeds its max context length of 4096.

[jfjensen]

I keep running into this issue as the max context length of my local LLM is 2048.

[robshox]

Any solution for this. Having the same problem with GPT4

[joaomdmoura]

Hey folks, we are working on a more native fix for this, probably a rolling context window, we needed to add the memory first before we could support it, but now that we have it we should be able to improve on this.

Sometime this could also indicated that you agent is trying to do too much and could be a good candidate to breaking into smaller agents / tasks.

That is not always de case tho.

[Abbie2020]

I came across this issue when running the sample code in Getting Started so you might want to include a simpler example that works.

[twelsh37]

Any update on this issue?

[rs8893]

The "Create agents to research and write an article" lesson in the Multi-agent Systems with crewAI Tutorial on deeplearning.ai Shorts works just fine with "HuggingFaceH4/zephyr-7b-beta". The markdown output has 2280 characters. But when I try the exact same notebook on Google Colab Pro and on my local Mac Mini M1 the output truncates at 485 characters without the markdown. Any suggestion for a fix? Thanks in advance.

[dhirajnair]

Any update on this @joaomdmoura . If there is native support to handle this , then an example would be helpful. Appreciate the good work on crewAI

[LeoAnt02]

Any solution for this ?

[lorenzejay]

We are actively developing this issue now. Will share the progress with everyone within the following week

[codebrain001]

Still looking forward to a resolution