crewAI
crewAI copied to clipboard
Fix no response / null response from LLM
This is a try solution for None / Empty response from LLM.
This PR assumes that this might be because of context length exceeded at the server side, which is not returning an error, just an empty response. This PR captures the ValueError which is being raised and try again with summarization.
[!NOTE] Detect and treat None/empty LLM responses as context-window overflows, trigger summarization, and retry; add targeted tests and minor typing updates.
- Agents (execution loop):
- Add
is_null_response_because_context_length_exceeded(...)alongsideis_context_length_exceeded(...)inCrewAgentExecutorandLiteAgentto detect empty responses likely caused by context overflow and invokehandle_context_length(summarization + retry).- Utilities:
- New
is_null_response_because_context_length_exceeded(exception, messages, llm)increwai/utilities/agent_utils.py(chunks messages byllm.get_context_window_size()and checks ValueError "None or empty").- Simplify
is_context_length_exceeded(exception)signature and docs; wireLLMMessagetyping in relevant modules.- Evaluator/Tools:
- Use
BaseAgentinTaskEvaluatorconstructor typing; importBaseAgentintool_utils.- Tests:
- Add tests for null/empty response handling path and for
is_null_response_because_context_length_exceededintests/utilities/test_agent_utils.py.- Extend agent tests to validate summarization and final answer flow when empty responses occur.
Written by Cursor Bugbot for commit 8b3484367762eef23e8fa8babe88f46482c996b7. This will update automatically on new commits. Configure here.
Hey @lucasgomide, do you think this could be possible solution for this? https://github.com/crewAIInc/crewAI/issues/2885#issuecomment-3338740438
I got some feedback from folks who tried the feature branch
- https://github.com/crewAIInc/crewAI/issues/2885#issuecomment-3337343477
- https://github.com/crewAIInc/crewAI/issues/2885#issuecomment-3337359639
Gentle ping on this one @lucasgomide
Added to my list to review later today
mind you checking the CI errors?
mind you checking the CI errors?
All fixed now.