[Bug]: CreateStreamedResponse fails with custom models that return sources without choices

[phanthai12]

Description

Bug: CreateStreamedResponse fails with custom models that return sources without choices

Issue Description

When using custom models through Open WebUI that return sources in the first response without a 'choices' array, the CreateStreamedResponse::from() method fails because it can't find the expected 'choices' key in the response attributes.

Steps To Reproduce

Current Behavior

The first response from a custom model using sources returns something like:

{
    "sources": [
        {
            "source": {
                "id": "b3d24ed7-a1f1-4054-bb37-55c9cf96ad70",
                "name": "Example Document",
                "type": "collection"
            },
            "document": ["Content example..."],
            "metadata": [
                {
                    "name": "Example File.txt",
                    "source": "Example File.txt"
                }
            ],
            "distances": [0.8338083668559383]
        }
    ]
}

This causes the following code in CreateStreamedResponse::from() to fail:

public static function from(array $attributes): self
{
    $choices = array_map(fn (array $result): CreateStreamedResponseChoice => CreateStreamedResponseChoice::from(
        $result
    ), $attributes['choices']);

    return new self(
        $attributes['id'],
        $attributes['object'],
        $attributes['created'],
        $attributes['model'],
        $choices,
        isset($attributes['usage']) ? CreateResponseUsage::from($attributes['usage']) : null,
    );
}

The error occurs because $attributes['choices'] doesn't exist in the response, causing the stream to fail instead of gracefully handling or skipping this response chunk.

Expected Behavior

The client should be able to handle response chunks that don't contain the 'choices' field by either:

1. Skipping chunks that don't have a 'choices' field, or
2. Adding support for the 'sources' field in the response format

This would allow the client to work with custom models that return sources information, particularly in the first chunk of a streamed response.

Environment

• Package version: v0.10.3
• PHP version: 8.3
• Using with: Open WebUI custom model with knowledges

OpenAI PHP Client Version

v0.10.3

PHP Version

8.3.16

Notes

This issue specifically affects custom models in Open WebUI that return sources information. The standard OpenAI API doesn't have this issue, but as more people use the client with alternative backends, handling non-standard response formats becomes important for compatibility.

[iBotPeaches]

I think thats an important thing we have to think about. Half the issues on the board are compatibility with a different platform, yet this is the OpenAI library.

I don't disagree with you, but all the typing becomes a bit moot if we are just skipping, null coalescing, etc our way through the minor differences on the other platforms. We'd have to make things nullable or missing and maybe thats not a bad thing, because I do see the benefit as all these other models have parity with OpenAI. Yet I don't want downstream users to have their linting tools go mad supporting all the null/changing aspects when it may not be relevant for OpenAI.

Yet when I think about a drop in replacement MinIO/Wasabi for Amazon S3 - it just works. Is it up to the Amazon PHP library to support alternatives that happen to have minor incompatibility with S3? Probably not.

I'd be curious if @nunomaduro / @gehrisandro have taken a stance one way or the other in terms of how they expect this library to adjust.

[phanthai12]

I appreciate the thoughtful response. I'd like to point out that the library already supports alternative backends through the withBaseUri() method:

$client = OpenAI::factory()
    ->withApiKey($config['api_key'])
    ->withBaseUri($config['base_uri'])
    ->withHttpClient($httpClient)
    ->withStreamHandler(fn (RequestInterface $request): ResponseInterface => $httpClient->send($request, [
      'stream' => true,
    ]))
    ->make();

This suggests the library was designed with some level of cross-platform compatibility in mind. Users are already encouraged to use different base URIs for alternative services that implement OpenAI-compatible APIs.

Given this existing flexibility, my suggestion for a compatibility mode aligns well with the library's architecture:

$client = OpenAI::factory()
    ->withApiKey($config['api_key'])
    ->withBaseUri($config['base_uri'])
    ->withCompatibilityMode('extended') // New method to handle response variations
    ->make();

This approach would:

1. Preserve the existing API design pattern
2. Acknowledge that withBaseUri() exists specifically to accommodate alternative backends
3. Extend that support to handle minor response variations without breaking type safety

By adding this option, you'd be completing the compatibility story that was already started with the base URI configuration, making the library truly useful for the growing ecosystem of OpenAI-compatible services while keeping the default behavior strict for pure OpenAI users.

[iBotPeaches]

I see. I wonder how an opt-in compatibility change could dynamically affect typing.

I believe historically the baseUrl feature was because OpenAI was available through Azure or directly from OpenAI - so you needed the ability to toggle between them. Of course that opened the gate for everything you see here.

[iBotPeaches]

After digging into this issue with Claude. I need a raw payload sample to confirm a fix. When I investigated this issue on Claude, it was a new property

data: {"type": "ping"}

So reducing the support on choices was not the right fix. So if you can provide a raw streamed response - just like do a (string) $response->getBody() before the library tries to parse it. You'll get full ugly payload

[iBotPeaches]

For context. Looking for something like this.

data: {"id":"msg_0111RgCFCqN68mJbev6Rq1cz","choices":[{"index":0,"delta":{"role":"assistant"}}],"created":1744469024,"model":"claude-3-7-sonnet-20250219","object":"chat.completion.chunk"}
data: {"type": "ping"}
data: {"id":"msg_0111RgCFCqN68mJbev6Rq1cz","choices":[{"index":0,"delta":{"content":"Hello!"}}],"created":1744469024,"model":"claude-3-7-sonnet-20250219","object":"chat.completion.chunk"}
data: {"id":"msg_0111RgCFCqN68mJbev6Rq1cz","choices":[{"index":0,"delta":{"content":" How can I assist you today? I'm here to help"}}],"created":1744469024,"model":"claude-3-7-sonnet-20250219","object":"chat.completion.chunk"}
data: {"id":"msg_0111RgCFCqN68mJbev6Rq1cz","choices":[{"index":0,"delta":{"content":" with information, answer questions, or discuss"}}],"created":1744469024,"model":"claude-3-7-sonnet-20250219","object":"chat.completion.chunk"}
data: {"id":"msg_0111RgCFCqN68mJbev6Rq1cz","choices":[{"index":0,"delta":{"content":" various topics. Feel free to let me know what you're"}}],"created":1744469024,"model":"claude-3-7-sonnet-20250219","object":"chat.completion.chunk"}
data: {"id":"msg_0111RgCFCqN68mJbev6Rq1cz","choices":[{"index":0,"delta":{"content":" interested in talking about."}}],"created":1744469024,"model":"claude-3-7-sonnet-20250219","object":"chat.completion.chunk"}
data: {"id":"msg_0111RgCFCqN68mJbev6Rq1cz","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"created":1744469024,"model":"claude-3-7-sonnet-20250219","object":"chat.completion.chunk"}
data: [DONE]

[phanthai12]

Sorry for the late reply, Because I used a regular HTTP library instead. Here is my data:

data: {"sources": [{"source": {"id": "48f6bceb-8f24-42b2-b144-df627b8bae17", "user_id": "776033a2-51da-488c-9155-56653bde5ec0", "name": "Documents", "description": "Documents", "meta": null, "access_control": null, "created_at": 1744181255, "updated_at": 1744181269, "user": {"id": "776033a2-51da-488c-9155-56653bde5ec0", "name": "Criss Anger", "email": "[email protected]", "role": "admin", "profile_image_url": "data:image/png;base64,..."}, "files": [{"id": "5442daa1-200e-4d2d-8f18-f9cac6016c5d", "meta": {"name": "rag-document.en.md", "content_type": "application/octet-stream", "size": 8644, "data": {}, "collection_name": "48f6bceb-8f24-42b2-b144-df627b8bae17"}, "created_at": 1744181268, "updated_at": 1744181268}, {"id": "114639b0-7c40-4f83-b2f4-3cec9746a2a8", "meta": {"name": "usage-rag.en.md", "content_type": "application/octet-stream", "size": 6114, "data": {}, "collection_name": "48f6bceb-8f24-42b2-b144-df627b8bae17"}, "created_at": 1744181265, "updated_at": 1744181265}], "type": "collection"}, "document": ["document details..."], "metadata": [{"created_by": "776033a2-51da-488c-9155-56653bde5ec0", "embedding_config": "{\"engine\": \"ollama\", \"model\": \"bge-m3:latest\"}", "file_id": "114639b0-7c40-4f83-b2f4-3cec9746a2a8", "hash": "dcf4e35eeb5ff66a673dc60aaec99611ed67f5eef5ecaa3051a202c4decfc5d2", "name": "usage-rag.en.md", "source": "usage-rag.en.md", "start_index": 0}, {"created_by": "776033a2-51da-488c-9155-56653bde5ec0", "embedding_config": "{\"engine\": \"ollama\", \"model\": \"bge-m3:latest\"}", "file_id": "114639b0-7c40-4f83-b2f4-3cec9746a2a8", "hash": "dcf4e35eeb5ff66a673dc60aaec99611ed67f5eef5ecaa3051a202c4decfc5d2", "name": "usage-rag.en.md", "source": "usage-rag.en.md", "start_index": 0}, {"created_by": "776033a2-51da-488c-9155-56653bde5ec0", "embedding_config": "{\"engine\": \"ollama\", \"model\": \"bge-m3:latest\"}", "file_id": "5442daa1-200e-4d2d-8f18-f9cac6016c5d", "hash": "f70571c5bcfcdc44b4aca1675f0f3abe2d099afe3f239d58fdc51f035a550106", "name": "rag-document.en.md", "source": "rag-document.en.md", "start_index": 0}, {"created_by": "776033a2-51da-488c-9155-56653bde5ec0", "embedding_config": "{\"engine\": \"ollama\", \"model\": \"bge-m3:latest\"}", "file_id": "5442daa1-200e-4d2d-8f18-f9cac6016c5d", "hash": "f70571c5bcfcdc44b4aca1675f0f3abe2d099afe3f239d58fdc51f035a550106", "name": "rag-document.en.md", "source": "rag-document.en.md", "start_index": 0}, {"created_by": "776033a2-51da-488c-9155-56653bde5ec0", "embedding_config": "{\"engine\": \"ollama\", \"model\": \"bge-m3:latest\"}", "file_id": "5442daa1-200e-4d2d-8f18-f9cac6016c5d", "hash": "f70571c5bcfcdc44b4aca1675f0f3abe2d099afe3f239d58fdc51f035a550106", "name": "rag-document.en.md", "source": "rag-document.en.md", "start_index": 0}]}]}

data: {"id": "gemma3:4b-28004c09-5f01-444b-bd55-e91f51961a6b", "created": 1745485619, "model": "gemma3:4b", "choices": [{"index": 0, "logprobs": null, "finish_reason": null, "delta": {"content": "1"}}], "object": "chat.completion.chunk"}

data: {"id": "gemma3:4b-06cf347e-e9f5-4d15-a3f0-a47664f77419", "created": 1745485619, "model": "gemma3:4b", "choices": [{"index": 0, "logprobs": null, "finish_reason": null, "delta": {"content": "."}}], "object": "chat.completion.chunk"}

data: {"id": "gemma3:4b-27314c41-5e2e-43fe-a7ce-5bdfe919ea53", "created": 1745485619, "model": "gemma3:4b", "choices": [{"index": 0, "logprobs": null, "finish_reason": null, "delta": {"content": "  "}}], "object": "chat.completion.chunk"}
... others
data: [DONE]

[iBotPeaches]

Sorry for delay. Going through old issues and looking at this fresh - this isn't like a partially filled out event. This is an entire new event that I cannot place with any matching OpenAI event.

data: {"sources": [{"source": {"id": "48f6bceb-8f24-42b2-b144-df627b8bae17", "user_id": "776033a2-51da-488c-9155-56653bde5ec0", "name": "Documents", "description": "Documents", "meta": null, "access_control": null, "created_at": 1744181255, "updated_at": 1744181269, "user": {"id": "776033a2-51da-488c-9155-56653bde5ec0", "name": "Criss Anger", "email": "[email protected]", "role": "admin", "profile_image_url": "data:image/png;base64,..."}, "files": [{"id": "5442daa1-200e-4d2d-8f18-f9cac6016c5d", "meta": {"name": "rag-document.en.md", "content_type": "application/octet-stream", "size": 8644, "data": {}, "collection_name": "48f6bceb-8f24-42b2-b144-df627b8bae17"}, "created_at": 1744181268, "updated_at": 1744181268}, {"id": "114639b0-7c40-4f83-b2f4-3cec9746a2a8", "meta": {"name": "usage-rag.en.md", "content_type": "application/octet-stream", "size": 6114, "data": {}, "collection_name": "48f6bceb-8f24-42b2-b144-df627b8bae17"}, "created_at": 1744181265, "updated_at": 1744181265}], "type": "collection"}, "document": ["document details..."], "metadata": [{"created_by": "776033a2-51da-488c-9155-56653bde5ec0", "embedding_config": "{\"engine\": \"ollama\", \"model\": \"bge-m3:latest\"}", "file_id": "114639b0-7c40-4f83-b2f4-3cec9746a2a8", "hash": "dcf4e35eeb5ff66a673dc60aaec99611ed67f5eef5ecaa3051a202c4decfc5d2", "name": "usage-rag.en.md", "source": "usage-rag.en.md", "start_index": 0}, {"created_by": "776033a2-51da-488c-9155-56653bde5ec0", "embedding_config": "{\"engine\": \"ollama\", \"model\": \"bge-m3:latest\"}", "file_id": "114639b0-7c40-4f83-b2f4-3cec9746a2a8", "hash": "dcf4e35eeb5ff66a673dc60aaec99611ed67f5eef5ecaa3051a202c4decfc5d2", "name": "usage-rag.en.md", "source": "usage-rag.en.md", "start_index": 0}, {"created_by": "776033a2-51da-488c-9155-56653bde5ec0", "embedding_config": "{\"engine\": \"ollama\", \"model\": \"bge-m3:latest\"}", "file_id": "5442daa1-200e-4d2d-8f18-f9cac6016c5d", "hash": "f70571c5bcfcdc44b4aca1675f0f3abe2d099afe3f239d58fdc51f035a550106", "name": "rag-document.en.md", "source": "rag-document.en.md", "start_index": 0}, {"created_by": "776033a2-51da-488c-9155-56653bde5ec0", "embedding_config": "{\"engine\": \"ollama\", \"model\": \"bge-m3:latest\"}", "file_id": "5442daa1-200e-4d2d-8f18-f9cac6016c5d", "hash": "f70571c5bcfcdc44b4aca1675f0f3abe2d099afe3f239d58fdc51f035a550106", "name": "rag-document.en.md", "source": "rag-document.en.md", "start_index": 0}, {"created_by": "776033a2-51da-488c-9155-56653bde5ec0", "embedding_config": "{\"engine\": \"ollama\", \"model\": \"bge-m3:latest\"}", "file_id": "5442daa1-200e-4d2d-8f18-f9cac6016c5d", "hash": "f70571c5bcfcdc44b4aca1675f0f3abe2d099afe3f239d58fdc51f035a550106", "name": "rag-document.en.md", "source": "rag-document.en.md", "start_index": 0}]}]}

This event doesn't have a object/type that other streamed events use. I peeked around Open WebUI docs and didn't see this documented anywhere. Perhaps the expansion for this is unknown events are just tossed with some warning emitted or something. So folks with configured error logging could ignore it - or not.