Error decoding empty object with optional fields

[sjbaldwin]

I believe there is currently a bug when trying to decode an empty object {} when the schema is for a case class with all Optional fields resulting in a expected '"' got '}' error.

Evidence:

Upgrading zio-openai from 0.4.0 to 0.4.1 resulted in a json decoding error for responses which decoded fine in the prior version (giving Encountered openai error: Unknown(zio.schema.codec.DecodeError$ReadError: .choices[0].delta(expected '"' got '}')) error on the final event of a chat response SSE stream). The json it was trying to decode was:

{"id":"chatcmpl-8qGJgtg8np7wvJuh1SfJRGY3SzLXn","object":"chat.completion.chunk","created":1707466468,"model":"gpt-3.5-turbo-0613","system_fingerprint":null,"choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}]}

With relevant part of schema:

final case class ChoicesItem(delta: ChatCompletionStreamResponseDelta, logprobs: Optional[CreateChatCompletionStreamResponse.ChoicesItem.Logprobs] = ???, finishReason: Optional[FinishReason], index: Int)

final case class ChatCompletionStreamResponseDelta(content: Optional[String] = ???, functionCall: Optional[ChatCompletionStreamResponseDelta.FunctionCall] = ???, toolCalls: Optional[Chunk[ChatCompletionMessageToolCallChunk]] = ???, role: Optional[ChatCompletionStreamResponseDelta.Role] = ???)

Further investigation determined that pinning dev.zio::zio-schema-json:0.4.17 to dev.zio::zio-schema-json:0.4.16 fixed the decoding issue. The commits going from 0.4.16 -> 0.4.17 can be found here.

Looking at those commits, I think the issue come from this line in #619.

Possible solution:

Current code in unsafeDecodeFields:

@tailrec
      def loop(index: Int, in: RetractReader): Unit = {
        val fieldRaw = Lexer.field(trace, in, new StringMatrix(aliasesMatrix))
        if (index == discriminator) {
          Lexer.skipValue(trace, in)
        } else {
          fieldRaw match {
            case -1 if index == discriminator => Lexer.skipValue(trace, in)
            case -1 if rejectExtraFields      => throw UnsafeJson(JsonError.Message("extra field") :: trace)
            case -1                           => Lexer.skipValue(trace, in)
            case idx =>
              val field = fieldAliases.getOrElse(aliasesMatrix(idx), -1)
              if (buffer(field) != null)
                throw UnsafeJson(JsonError.Message("duplicate") :: trace)
              else
                buffer(field) = schemaDecoder(schemas(field)).unsafeDecode(spans(field) :: trace, in)
          }
        }
        if (Lexer.nextField(trace, in)) loop(index + 1, in)
      }

      if (discriminator == -1) {
        Lexer.char(trace, in, '{')
        loop(0, in)
      } else if (discriminator == -2) {
        if (Lexer.nextField(trace, in)) loop(0, in)
      } else {
        val rr = RecordingReader(in)
        if (Lexer.firstField(trace, rr)) loop(0, rr)
      }

If the discriminator is -1 (meaning no discriminator), then the Lexer reads in the { and immediately starts the loop. The loop calls Lexer.field(trace, in, new StringMatrix(aliasesMatrix)) which immediately tries reading in a field (starting with " ) and throws an error if it doesnt see it. I think what we need to do is change the code to:

if (discriminator == -1) {
    Lexer.char(trace, in, '{')
    if(Lexer.firstField(trace, in)) loop(0, in)
}