Feature request: default handling of undefined values in the standard lib json module

[erlend-axelsson]

Is your feature request related to a problem? Please describe. I have a use case where I need to send a json object with an explicit undefined value, and I discovered that the default behavior of json:encode/1 and json:decode is to encode the atom 'undefined' as a string, and throw an exception while decoding. While it isn't a massive issue I believe it would be a quality of life improvement to treat 'undefined' as a special case like how 'null' is.

Describe the solution you'd like Add a special case where 'undefined' gets (en/de)coded as undefined like how 'null' gets (en/de)coded as null in the default json:encode and decode functions.

Describe alternatives you've considered Alternatively providing an encoder and decoder like json:encode_key_value_list_checked/2 which handles 'undefined' as undefined would also work if changing the defaults risks introducing regressions in existing code.

[erlend-axelsson]

I read up on the RFC 8259 and ECMA 404 standards, so it seems I was wrong in believing undefined to be a part of the JSON standard.

I still think there is some value in at least allowing the option of a "lax" encoding/decoding since it's not terribly uncommon for JSON "in the wild" to contain undefined values. It was relatively easy to get json:encode/2 to encode undefined, however it seems that the decoding part is hard coded to reject undefined as decode/1, decode/3 and decode_start/3 all immediately call json:value/6 which then errors on the function overload at line 797.

ref encoding undefined:

custom_encode(undefined, _Encode) ->
    <<"undefined">>;
custom_encode(Value, Encode) ->
    json:encode_value(Value, Encode).
custom_encode(Value) -> json:encode(Value, fun custom_encode/2).

[josevalim]

There is always loss of information when encoding atoms to JSON:

2> erlang:iolist_to_binary(json:encode(#{foo => bar})).
<<"{\"foo\":\"bar\"}">>

If we convert the JSON string "undefined" into atom, it will be the only string that gets converted to an atom. Doing it automatically may also cause it to pop-up on undefined places: someone may literally type their name as "undefined" as their username, but in that case, it is not really undefined!

If you are expecting "undefined" in some places, it probably makes the most sense to do it for certain keys, as a post-processing pass on the data?

[dgud]

@josevalim strings are handled and can be overridden by the user, it is the the set of "atoms" (true, false and null) in json that can't be extended by the user.

But I'm closing this for now, not part of the standard and this would kind of kill the performance in the hot part of decode.