openai-node icon indicating copy to clipboard operation
openai-node copied to clipboard

Cannot stream chat completions from Azure

Open johanbaath opened this issue 6 months ago • 9 comments

Confirm this is a Node library issue and not an underlying OpenAI API issue

  • [X] This is an issue with the Node library

Describe the bug

When streaming chat completions using client.chat.completions.create with AzureOpenAI client and reading with ChatCompletionStreamingRunner.fromReadableStream on the client, the following error occurs:

OpenAIError: Cannot read properties of undefined (reading 'content')

Cause:

The error seems to be caused by choice.delta being undefined at some point during the streaming process, usually at the end of the stream.

Questions:

  1. Is this a known issue?
  2. Will this library support streaming from Azure, given potential differences in response structure?

To Reproduce

  1. Initialize an AzureOpenAI client
  2. Using client.chat.completions.create stream a response to the client (a web app)
  3. Stream the response using ChatCompletionStreamingRunner.fromReadableStream on the client
  4. Observe the error

Code snippets

No response

OS

macOS

Node version

Node v20.14.0

Library version

openai 4.56.0

johanbaath avatar Aug 27 '24 18:08 johanbaath