Fix bug with Azure Async filter events processing
- [x] I understand that this repository is auto-generated and my pull request may not be merged
Changes being requested
When using (Async)ChatCompletionStream, add a weak validation before the accumulation, to ignore events not adhering to the ChatCompletionChunk schema.
This prevents runtime errors when the SSEs are not 100% compatible with OpenAI spec (e.g. when using Azure OpenAI with Asynchronous Filter feature).
Before this fix, the following code will result in an error:
from openai import AsyncAzureOpenAI
client = AsyncAzureOpenAI(
api_version="2024-08-01-preview",
azure_endpoint="<azure openai endpoint !!! with Asynchronous Filter enabled !!!>",
api_key="<api key>",
)
async with client.beta.chat.completions.stream(
model="deployment-name",
messages=[
{
"role": "user",
"content": "hi",
},
],
) as stream:
async for event in stream:
print(event)
# ...
# ContentDoneEvent(...)
# then Exception:
File ~/<path>/openai/lib/streaming/chat/_completions.py:397, in ChatCompletionStreamState._accumulate_chunk(self, chunk)
369 choice_snapshot = completion_snapshot.choices[choice.index]
370 previous_tool_calls = choice_snapshot.message.tool_calls or []
372 choice_snapshot.message = cast(
373 ParsedChatCompletionMessageSnapshot,
374 construct_type(
375 type_=ParsedChatCompletionMessageSnapshot,
376 value=accumulate_delta(
377 cast(
378 "dict[object, object]",
379 model_dump(
380 choice_snapshot.message,
381 # we don't want to serialise / deserialise our custom properties
382 # as they won't appear in the delta and we don't want to have to
383 # continuosly reparse the content
384 exclude=cast(
385 # cast required as mypy isn't smart enough to infer `True` here to `Literal[True]`
386 IncEx,
387 {
388 "parsed": True,
389 "tool_calls": {
390 idx: {"function": {"parsed_arguments": True}}
391 for idx, _ in enumerate(choice_snapshot.message.tool_calls or [])
392 },
393 },
394 ),
395 ),
396 ),
--> 397 cast("dict[object, object]", choice.delta.to_dict()),
398 ),
399 ),
400 )
402 # ensure tools that have already been parsed are added back into the newly
403 # constructed message snapshot
404 for tool_index, prev_tool in enumerate(previous_tool_calls):
AttributeError: 'NoneType' object has no attribute 'to_dict'
Additional context & links
The primary purpose of this PR is to make this SDK usable with Azure OpenAI Asynchronous Filter feature.
E.g. Langchain internally relies on client.beta.chat.completions.stream in some cases.
An issue describing non-compatible events when using an Asynchronous Feature:
- https://github.com/openai/openai-python/issues/1677
Fixes to the same problem in other popular SDK's:
- https://github.com/wandb/weave/pull/3922
- https://github.com/run-llama/llama_index/pull/16636
Other possibly related issues:
- https://github.com/langgenius/dify/issues/5790
- https://github.com/TheR1D/shell_gpt/issues/662
CC @kristapratico