adk-python icon indicating copy to clipboard operation
adk-python copied to clipboard

fix(models): Handle empty message in LiteLLM response (fixes #3618)

Open sarojrout opened this issue 1 month ago • 4 comments

When a turn ends with only tool calls and no final agent message, _model_response_to_generate_content_response was raising ValueError. This fix returns an empty LlmResponse instead, allowing workflows to continue gracefully.

Changes:

  • Return empty LlmResponse when message is None or empty
  • Preserve finish_reason and usage_metadata if available
  • Added test cases for the edge cases

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

  • Closes: #3618
  • Related: #3618

2. Or, if no issue exists, describe the change:

If applicable, please follow the issue templates to provide as much detail as possible.

Problem: When using google-adk with LiteLLM and tools, a run can end with a tool call/tool response and no final agent message. In these cases, the LiteLLM wrapper was raising ValueError: No message in response from _model_response_to_generate_content_response, causing workflows to crash even though this is a valid scenario.

This is particularly problematic in multi-agent workflows where:

  • A turn can legitimately end with only tool calls (no text message)
  • The LLM model may write intermediate responses and finalize the turn by calling a tool
  • Integration with observability frameworks marks these as errors even though the system is working correctly

Solution: Instead of raising ValueError when message is None or empty, the code now:

  1. Creates an empty LlmResponse with empty content parts
  2. Preserves finish_reason if available (properly mapped to the appropriate enum)
  3. Preserves usage_metadata if available
  4. Returns the response gracefully, allowing workflows to continue

This solution treats a turn ending with only tool calls as a valid outcome, which aligns with the expected behavior described in the issue. The fix is backward-compatible - existing code that works will continue to work, and the edge case is now handled gracefully.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.

Unit Tests:

  • [x] I have added or updated unit tests for my change.
  • [x] All unit tests pass locally.

Added three new test cases in tests/unittests/models/test_litellm.py:

  1. test_model_response_to_generate_content_response_no_message_with_finish_reason

    • Tests response with no message but has finish_reason="tool_calls"
    • Verifies empty LlmResponse is returned with correct finish_reason and usage_metadata
  2. test_model_response_to_generate_content_response_no_message_no_finish_reason

    • Tests response with no message and no finish_reason
    • Verifies empty LlmResponse is returned without errors
  3. test_model_response_to_generate_content_response_empty_message_dict

    • Tests response with empty message dict (message = {})
    • Verifies empty LlmResponse is returned correctly

Test Results:


$ pytest tests/unittests/models/test_litellm.py -k "test_model_response_to_generate_content_response_no_message" -v

tests/unittests/models/test_litellm.py::test_model_response_to_generate_content_response_no_message_with_finish_reason PASSED [ 50%]
tests/unittests/models/test_litellm.py::test_model_response_to_generate_content_response_no_message_no_finish_reason PASSED [100%]
================================ 2 passed, 102 deselected in 5.69s ================================
tests/unittests/models/test_litellm.py::test_model_response_to_generate_content_response_empty_message_dict PASSED [100%]

================================ 1 passed, 103 deselected in 5.71s ==========================

All existing tests pass:

$ pytest tests/unittests/models/test_litellm.py -v
=============================104 passed, 4 warnings in 6.40s =================================```

Manual End-to-End (E2E) Tests:

I have tested with the below utility code which you can try to test out

from litellm import ChatCompletionAssistantMessage
from litellm.types.utils import ModelResponse
from google.adk.models.lite_llm import _model_response_to_generate_content_response

print("=" * 60)

# Test 1: Response WITHOUT message (the bug scenario)
print("\n Test 1: Response WITHOUT message (bug scenario)")
print("-" * 60)
response_no_message = ModelResponse(
    model="test",
    choices=[{
        "finish_reason": "tool_calls",
        # message is missing - this used to raise ValueError
    }],
    usage={
        "prompt_tokens": 10,
        "completion_tokens": 5,
        "total_tokens": 15,
    },
)

try:
    result1 = _model_response_to_generate_content_response(response_no_message)
    print("SUCCESS: No ValueError raised!")
    print(f"   finish_reason: {result1.finish_reason}")
    print(f"   content parts: {len(result1.content.parts)}")
    print(f"   usage tokens: {result1.usage_metadata.total_token_count}")
    print("  Empty LlmResponse returned correctly")
except ValueError as e:
    if "No message in response" in str(e):
        print(f"FAILED: Still raises ValueError: {e}")
        print("   The fix is NOT working!")
    else:
        raise

# Test 2: Response WITH message (normal case - should still work)
print("\n Test 2: Response WITH message (normal case)")
print("-" * 60)
response_with_message = ModelResponse(
    model="test",
    choices=[{
        "message": ChatCompletionAssistantMessage(
            role="assistant",
            content="This is a normal response with text",
        ),
        "finish_reason": "stop",
    }],
    usage={
        "prompt_tokens": 10,
        "completion_tokens": 8,
        "total_tokens": 18,
    },
)

try:
    result2 = _model_response_to_generate_content_response(response_with_message)
    print("SUCCESS: Normal case still works!")
    print(f"   finish_reason: {result2.finish_reason}")
    print(f"   content parts: {len(result2.content.parts)}")
    if result2.content.parts:
        print(f"   text content: {result2.content.parts[0].text}")
    print(f"   usage tokens: {result2.usage_metadata.total_token_count}")
    print(" Normal message processing works correctly")
except Exception as e:
    print(f"FAILED: Error in normal case: {e}")
    raise

# Test 3: Response WITH message AND tool calls
print("\n Test 3: Response WITH message AND tool calls")
print("-" * 60)
from litellm import ChatCompletionMessageToolCall, Function

response_with_tool_calls = ModelResponse(
    model="test",
    choices=[{
        "message": ChatCompletionAssistantMessage(
            role="assistant",
            content="I'll call a tool for you",
            tool_calls=[
                ChatCompletionMessageToolCall(
                    type="function",
                    id="call_123",
                    function=Function(
                        name="test_function",
                        arguments='{"arg": "value"}',
                    ),
                )
            ],
        ),
        "finish_reason": "tool_calls",
    }],
    usage={
        "prompt_tokens": 10,
        "completion_tokens": 12,
        "total_tokens": 22,
    },
)

try:
    result3 = _model_response_to_generate_content_response(response_with_tool_calls)
    print("SUCCESS: Tool calls case works!")
    print(f"   finish_reason: {result3.finish_reason}")
    print(f"   content parts: {len(result3.content.parts)}")
    for i, part in enumerate(result3.content.parts):
        if part.text:
            print(f"   part {i}: text = '{part.text}'")
        if part.function_call:
            print(f"   part {i}: function_call = {part.function_call.name}")
    print(f"   usage tokens: {result3.usage_metadata.total_token_count}")
    print("Tool calls processing works correctly")
except Exception as e:
    print(f"FAILED: Error in tool calls case: {e}")
    raise

print("\n" + "=" * 60)

Checklist

  • [x] I have read the CONTRIBUTING.md document.
  • [x] I have performed a self-review of my own code.
  • [x] I have commented my code, particularly in hard-to-understand areas. - Added docstring explaining the behavior when message is None - Added inline comments explaining the fix logic
  • [x] I have added tests that prove my fix is effective or that my feature works.
  • [x] New and existing unit tests pass locally with my changes. - All 104 existing tests pass - All 3 new tests pass
  • [x] I have manually tested my changes end-to-end.
  • [x] Any dependent changes have been merged and published in downstream modules.
    • No dependent changes required

Additional context

Code Changes:

  • Modified src/google/adk/models/lite_llm.py::_model_response_to_generate_content_response()
  • Changed from raising ValueError to returning empty LlmResponse when message is None/empty
  • Preserves all available metadata (finish_reason, usage_metadata) in the empty response

sarojrout avatar Nov 24 '25 22:11 sarojrout

Summary of Changes

Hello @sarojrout, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves an issue in the google-adk LiteLLM integration where the system would incorrectly raise a ValueError when an LLM model's response contained no explicit message, particularly in cases where a turn concluded with only tool calls. This behavior, though valid in multi-agent workflows, caused crashes and misinterpretations as errors. The solution modifies the response processing to return an empty LlmResponse gracefully, preserving critical metadata like finish_reason and usage_metadata, thereby ensuring workflow continuity and accurate system behavior.

Highlights

  • Graceful Error Handling: The _model_response_to_generate_content_response function now gracefully handles LiteLLM responses where the message is None or empty, preventing ValueError exceptions that previously caused crashes.
  • Empty LlmResponse Return: Instead of raising an error, the function now returns an empty LlmResponse when no message is present, which is a valid outcome for turns ending with only tool calls in multi-agent workflows.
  • Metadata Preservation: Important metadata such as finish_reason and usage_metadata are now correctly preserved and included in the returned empty LlmResponse, ensuring no loss of critical information.
  • Enhanced Test Coverage: New unit tests have been added to specifically cover scenarios where responses lack messages (including None, missing, or empty dictionary messages), ensuring the robustness and correctness of the fix.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

[^1]: Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

gemini-code-assist[bot] avatar Nov 24 '25 22:11 gemini-code-assist[bot]

Hi @sarojrout ,Thank you for your work on this pull request. We appreciate the effort you've invested. Before we can proceed with the review can you please fix the lint errors. You can use autoformat.sh.

ryanaiagent avatar Nov 30 '25 05:11 ryanaiagent

Hi @sarojrout ,Thank you for your work on this pull request. We appreciate the effort you've invested. Before we can proceed with the review can you please fix the lint errors. You can use autoformat.sh.

sure will do it in sometime. thanks for reviewing @ryanaiagent

sarojrout avatar Nov 30 '25 05:11 sarojrout

Hi @sarojrout ,Thank you for your work on this pull request. We appreciate the effort you've invested. Before we can proceed with the review can you please fix the lint errors. You can use autoformat.sh.

sure will do it in sometime. thanks for reviewing @ryanaiagent

pls check @ryanaiagent whenever you get time. thanks!

sarojrout avatar Nov 30 '25 05:11 sarojrout