microsoft
Python: Azure OpenAI returns empty delta at the end of the completion
Context
Fix #7114.
Contribution checklist
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
Python 3.10 Test Coverage Report •
File Stmts Miss Cover Missing semantic_kernel/connectors/ai/open_ai/services azure_chat_completion.py 85 32 62% 141, 147–148, 157–158, 163–180, 184–188, 200–211 open_ai_chat_completion_base.py 168 70 58% 100, 104, 124, 149–153, 178, 186, 188, 192, 210–215, 233–261, 264–275, 290–297, 308–316, 332–339, 360, 368, 374–377, 389–392, 423 TOTAL 6648 762 89%
Python 3.10 Unit Test Overview
| Tests | Skipped | Failures | Errors | Time |
|---|---|---|---|---|
| 1575 | 1 :zzz: | 0 :x: | 0 :fire: | 19.359s :stopwatch: |
Hi @clemlesne, this change has caused many of the integration tests to fail. Can you have a look, please? https://github.com/microsoft/semantic-kernel/actions/runs/9840286760/job/27164184698#step:7:1501
Hi @clemlesne, I've reached out to a contact on the Azure OpenAI team to make sure this isn't a bug. Will let you know when I hear back (ETA early next week).
Can this Pull Request be accepted if I resolve the conflicts...?
If it can be accepted, I intend to resolve the conflicts myself.(However, I believe you've also contacted the Azure OpenAI team, so I plan to confirm before proceeding with the work.)
Can this Pull Request be accepted if I resolve the conflicts...?
If it can be accepted, I intend to resolve the conflicts myself.(However, I believe you've also contacted the Azure OpenAI team, so I plan to confirm before proceeding with the work.)
Hi @yuichiromukaiyama, thank you for following up. I received the following response from a colleague:
"As far as I can tell, this is the intended behavior - you can see in AOAI public docs the example stream
Azure OpenAI Service content filtering - Azure OpenAI | Microsoft Learn does not have delta in the last event before [DONE]. I reproduced this myself calling the API directly as well. I think the nature of the async content filter is such that the content checking is done separately from the chat stream, so there is no delta associated with the async content filter results, rather it references a portion of the previous stream via
content_filter_offsets.
So, the chat completion would be streaming back with deltas, meanwhile content filter is asynchronously analyzing the stream, the content finishes first (last delta is {}), then the last content filter messages come through, then the stream finishes."
It does seem we would want the behavior in this PR.
@moonbox3 Thank you! I have issued a pull request with resolved conflicts. I found some parts of the code that could be improved, so I rewrote them in a separate pull request.
https://github.com/microsoft/semantic-kernel/pull/8075