crewAI icon indicating copy to clipboard operation
crewAI copied to clipboard

Fix no response / null response from LLM

Open Vidit-Ostwal opened this issue 2 months ago • 5 comments

This is a try solution for None / Empty response from LLM.

This PR assumes that this might be because of context length exceeded at the server side, which is not returning an error, just an empty response. This PR captures the ValueError which is being raised and try again with summarization.


[!NOTE] Detect and treat None/empty LLM responses as context-window overflows, trigger summarization, and retry; add targeted tests and minor typing updates.

  • Agents (execution loop):
    • Add is_null_response_because_context_length_exceeded(...) alongside is_context_length_exceeded(...) in CrewAgentExecutor and LiteAgent to detect empty responses likely caused by context overflow and invoke handle_context_length (summarization + retry).
  • Utilities:
    • New is_null_response_because_context_length_exceeded(exception, messages, llm) in crewai/utilities/agent_utils.py (chunks messages by llm.get_context_window_size() and checks ValueError "None or empty").
    • Simplify is_context_length_exceeded(exception) signature and docs; wire LLMMessage typing in relevant modules.
  • Evaluator/Tools:
    • Use BaseAgent in TaskEvaluator constructor typing; import BaseAgent in tool_utils.
  • Tests:
    • Add tests for null/empty response handling path and for is_null_response_because_context_length_exceeded in tests/utilities/test_agent_utils.py.
    • Extend agent tests to validate summarization and final answer flow when empty responses occur.

Written by Cursor Bugbot for commit 8b3484367762eef23e8fa8babe88f46482c996b7. This will update automatically on new commits. Configure here.

Vidit-Ostwal avatar Sep 25 '25 19:09 Vidit-Ostwal

Hey @lucasgomide, do you think this could be possible solution for this? https://github.com/crewAIInc/crewAI/issues/2885#issuecomment-3338740438

I got some feedback from folks who tried the feature branch

  • https://github.com/crewAIInc/crewAI/issues/2885#issuecomment-3337343477
  • https://github.com/crewAIInc/crewAI/issues/2885#issuecomment-3337359639

Vidit-Ostwal avatar Sep 26 '25 18:09 Vidit-Ostwal

Gentle ping on this one @lucasgomide

Vidit-Ostwal avatar Oct 30 '25 12:10 Vidit-Ostwal

Added to my list to review later today

lucasgomide avatar Oct 30 '25 12:10 lucasgomide

mind you checking the CI errors?

lucasgomide avatar Oct 30 '25 14:10 lucasgomide

mind you checking the CI errors?

All fixed now.

Vidit-Ostwal avatar Oct 30 '25 14:10 Vidit-Ostwal