openai-node icon indicating copy to clipboard operation
openai-node copied to clipboard

fix: Leading whitespace chunks break partial parser (structured outputs)

Open tillkolter opened this issue 10 months ago • 2 comments

  • [x] I understand that this repository is auto-generated and my pull request may not be merged

Changes being requested

Although gpt-4o-mini-2024-07-18 in general supports structured outputs, we observed after integration that the model responds quite regularly with leading newline chunks (while the overall response is valid JSON)

Since the partial parser does not allow (trimmed) empty strings – which is also reasonable – the only reasonable thing to handle such case is to handle this case before the parser is tried and default to a null value instead.

Additional context & links

Example Fingerprint: fp_0705bf87c0 Request ID: chatcmpl-Ab7XYB0hNxuJawO9gM0c4Doy0GTQP

Beginning of example output stream

data: {"id":"chatcmpl-Ab7XYB0hNxuJawO9gM0c4Doy0GTQP","object":"chat.completion.chunk","created":1733410484,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0705bf87c0","choices":[{"index":0,"delta":{"role":"assistant","content":"","refusal":null},"logprobs":null,"finish_reason":null}],"usage":null}

data: {"id":"chatcmpl-Ab7XYB0hNxuJawO9gM0c4Doy0GTQP","object":"chat.completion.chunk","created":1733410484,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0705bf87c0","choices":[{"index":0,"delta":{"content":"\n\n"},"logprobs":null,"finish_reason":null}],"usage":null}

data: {"id":"chatcmpl-Ab7XYB0hNxuJawO9gM0c4Doy0GTQP","object":"chat.completion.chunk","created":1733410484,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0705bf87c0","choices":[{"index":0,"delta":{"content":"{\""},"logprobs":null,"finish_reason":null}],"usage":null}
...

tillkolter avatar Dec 05 '24 15:12 tillkolter