Why "responses" endpoint are not visualized as "completions" endpoint with litellm wrapper

[anotine10]

Question

i have tried instantiating a litellm router or directly calls litellm "responses" endpoint. logfire does not show the information as nice as it if were the "completion" endpoint.

Example with "responses" endpoint:

• no input/output token are extracted, no UI for message transcript.

Good Example with completion endpoint:

As a note, it works well if i use the "response" endpoint from Openai or AzureOPenai object (without using litellm)

[alexmojaki]

Your screenshot shows a custom logfire log. Did you mean to show the responses span above?

Please share example code.

Chances are this just isn't implemented.

If you're using a litellm proxy, try talking to it with an instrumented openai client instead of the litellm sdk.

[anotine10]

hi @alexmojaki ,

i paste here a simple snippet code to reproduce:

import logfire
from pydantic import BaseModel, Field
from typing import Any, Type
import re
from openai import pydantic_function_tool
from litellm import Router

logfire.instrument_openai()
logfire.instrument_litellm()

#### > utility
def get_openai_tool_config_from_pydantic(
    model: Type[BaseModel],
    name: str | None = None,
    description: str | None = None,
    use_responses_api: bool = False,
) -> dict[str, Any]:
    """Get the tool config for the OpenAI from a Pydantic model

    Args:
        model: The Pydantic model to create the config from
        name: The name of the tool
        description: The description of the tool
        use_responses_api: If True, use the Responses API instead of the Completions API

    OpenAI Responses API uses different JSON schema for the tool config than the Completions API.

    Returns:
        The tool config for the OpenAI
    """
    if not name:
        name = re.sub(r"(?<!^)(?=[A-Z])", "_", model.__name__).lower()

    if not description:
        description = (model.__doc__ or "").strip()

    tool_config = pydantic_function_tool(
        model=model, name=name, description=description
    )

    if use_responses_api:
        return get_upgraded_tool_definition(tool_config)
    else:
        return dict(tool_config)


def get_upgraded_tool_definition(legacy_tool_definition):
    copy = legacy_tool_definition.copy()
    base = {
        "type": "function",
        "name": copy["function"]["name"],
        "description": copy["function"]["description"],
    }
    parameters = copy["function"]["parameters"]
    if "description" in parameters:
        del parameters["description"]

    base["parameters"] = parameters
    return base


def get_litellm_client():
        model_list = [
            {  # list of model deployments
                "model_name": "gpt-4.1",
                "litellm_params": {  # params for litellm completion/embedding call
                    "model": "azure/gpt-4.1",  # actual model name
                    "api_key": "YOUR APIKEY",
                    "api_version": "2025-03-01-preview",
                    "api_base": "YOUR API BASE",
                    "base_model": "azure/gpt-4.1",
                },
            }
        ]

        router = Router(
            model_list=model_list,
            model_group_alias={"gpt-4-1": "gpt-4-1"},
        )
        return router

#### < utility


system_prompt = """you a are a helpful assistant extracting information from user input"""
user_prompt = """Extract the key points from the following text: 'I am in londres'"""
class ExtractLocation(BaseModel):
    location: str = Field(..., description="The extracted location from the user input")

tools = [get_openai_tool_config_from_pydantic(ExtractLocation, "extract_location", use_responses_api=True)]


def test_litellm_openai_responses():
    client_openai = get_litellm_client()
    response = client_openai.responses(
        model="gpt-4.1",
        instructions=system_prompt,
        input=user_prompt,
        temperature=1,
        tools=tools,
        tool_choice="required",
    )
    print("Response:", response)

def test_litellm_openai_completion():
    client_openai = get_litellm_client()
    response = client_openai.completion(
        model="gpt-4.1",
        messages=[{"role": "user", "content": "Hey, how's it going?"}]
    )
    print("Response:", response)
    return response

test_litellm_openai_responses() # This one is not handled nicely in logfire
test_litellm_openai_completion() # this one is handled correctly in logfire

[anotine10]

Regarding your last message, what do you mean by If you're using a litellm proxy, try talking to it with an instrumented openai client instead of the litellm sdk. ? Thanks

[alexmojaki]

Regarding your last message, what do you mean by If you're using a litellm proxy, try talking to it with an instrumented openai client instead of the litellm sdk. ? Thanks

Ignore that, since you're using the litellm SDK. You mentioned 'router' and I thought that meant the proxy.

Reduced example:

import litellm

import logfire

logfire.configure()
logfire.instrument_litellm()

litellm.responses(model='gpt-4o-mini', input='Hi')

[anotine10]

Thanks @alexmojaki . But i am confused. what do you mean with your reduced example?

i do not know if you seen it, just before the message you replied, i have put a snippet of code that shows the difference between how logfire handle repsonses and completion endpoint from litellm.

[alexmojaki]

The reduced example is a minimal repro. Your snippet is more complicated than it needs to be.

[anotine10]

@alexmojaki , how to know when this feature will enter in logfire roadmap? Thanks

[alexmojaki]

Hard to say, if you're a paying customer or are willing to open a PR (I could give guidance) it would help