golang
encoding/json/v2: Decoder.More() == false on syntax error
dec := json.NewDecoder(strings.NewReader(","))
fmt.Println(dec.More())
fmt.Println(dec.Token())
$ go run .
true
<nil> invalid character ',' looking for beginning of value
$ GOEXPERIMENT=jsonv2 go run .
false
<nil> invalid character ',' looking for beginning of value
The JSONv1 implementation of More reports false if there is an error filling its buffer, but it reports true if the next token is semantically invalid. JSONv2 appears to report false if the next token is invalid.
Related Issues
- encoding/json/v2: change in Decoder.Token behavior at unexpected EOF #74750 (closed)
- encoding/json: don't silently ignore errors from (*Decoder).More #67525
- encoding/json: GOEXPERIMENT=jsonv2 changes behavior of Decoder.InputOffset after More #75468 (closed)
- encoding/json: unmarshal regression for extra data error under goexperiment.jsonv2 #74614 (closed)
- encoding/json/v2: fails to return SyntacticError on empty input #74548 (closed)
- encoding/json: Decoder.Token does not return an error for incomplete JSON #69449
- encoding/json/v2: UnmarshalTypeError text changes #74713 (closed)
Related Code Changes
- encoding/json: don't silently ignore errors from (*Decoder).More
- encoding/json: add Token for streaming parse
(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)
Interesting. I don't know why it behaves this way, but the observable behavior seems to go against the intent of the v1 code: https://github.com/golang/go/blob/5945fc02fc575da1edc80bcdd0e88e41689552a7/src/encoding/json/stream.go#L485-L488
jsontext.Decoder.PeekKind returns Kind(0) on both error and EOF. This makes it impossible to distinguish between an error and the natural end of input without reading the next token.
Maybe that's okay, but would it perhaps make sense to have separate Kinds for EOF and error?
Change https://go.dev/cl/728300 mentions this issue: encoding/json: report true from v2 Decoder.More when an error is pending
