extensions icon indicating copy to clipboard operation
extensions copied to clipboard

GetResponseAsync_StructuredOutput_NonNative fails with JsonException : '`' is an invalid start of a value.

Open ericstj opened this issue 2 weeks ago • 0 comments

https://github.com/dotnet/extensions/blob/cc328a764d1b0200a852772b1f163ebecc7f4a22/test/Libraries/Microsoft.Extensions.AI.Integration.Tests/ChatClientIntegrationTests.cs#L1092-L1117

Microsoft.Extensions.AI.OpenAIAssistantChatClientIntegrationTests.GetResponseAsync_StructuredOutput_NonNative will occasionally fail with the following exception:

Microsoft.Extensions.AI.OpenAIAssistantChatClientIntegrationTests.GetResponseAsync_StructuredOutput_NonNative [FAIL]
  System.Text.Json.JsonException : '`' is an invalid start of a value. Path: $ | LineNumber: 0 | BytePositionInLine: 0.
  ---- System.Text.Json.JsonReaderException : '`' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.
  Stack Trace:
       at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, JsonReaderException ex)
       at System.Text.Json.JsonSerializer.GetReaderScopedToNextValue(Utf8JsonReader& reader, ReadStack& state)
       at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
       at System.Text.Json.JsonSerializer.Deserialize[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
    C:\src\dotnet\extensions\src\Libraries\Microsoft.Extensions.AI\ChatCompletion\ChatResponse{T}.cs(107,0): at Microsoft.Extensions.AI.ChatResponse`1.DeserializeFirstTopLevelObject(String json, JsonTypeInfo`1 typeInfo)
    C:\src\dotnet\extensions\src\Libraries\Microsoft.Extensions.AI\ChatCompletion\ChatResponse{T}.cs(124,0): at Microsoft.Extensions.AI.ChatResponse`1.GetResultCore(Nullable`1& failureReason)
    C:\src\dotnet\extensions\src\Libraries\Microsoft.Extensions.AI\ChatCompletion\ChatResponse{T}.cs(60,0): at Microsoft.Extensions.AI.ChatResponse`1.get_Result()
    C:\src\dotnet\extensions\test\Libraries\Microsoft.Extensions.AI.Integration.Tests\ChatClientIntegrationTests.cs(1109,0): at Microsoft.Extensions.AI.ChatClientIntegrationTests.GetResponseAsync_StructuredOutput_NonNative()
    --- End of stack trace from previous location ---
    ----- Inner Stack Trace -----
       at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
       at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker)
       at System.Text.Json.Utf8JsonReader.ReadFirstToken(Byte first)
       at System.Text.Json.Utf8JsonReader.ReadSingleSegment()
       at System.Text.Json.Utf8JsonReader.Read()
       at System.Text.Json.JsonSerializer.GetReaderScopedToNextValue(Utf8JsonReader& reader, ReadStack& state)

I can make it fail about 2 out of 5 times. When it fails I debugged to see what the model returned. It looks like it returned a markdown block instead of raw JSON. The response was the following.

```json
{
  "fullName": "Jimbo Smith",
  "ageInYears": 34,
  "homeTown": "Cardiff",
  "job": "Programmer"
}
```

ericstj avatar Dec 12 '25 19:12 ericstj