jsoniter-scala icon indicating copy to clipboard operation
jsoniter-scala copied to clipboard

Consider adding support of UTF-16 & UTF-32 binaries as input/output

Open plokhotnyuk opened this issue 7 years ago • 2 comments

The IETF just pub­lished RFC 8259 (al­so known as “STD 90”): https://www.rfc-editor.org/rfc/rfc8259.txt It con­tains revised section about encoding:

8.1.  Character Encoding

   JSON text exchanged between systems that are not part of a closed
   ecosystem MUST be encoded using UTF-8 [RFC3629].

   Previous specifications of JSON have not required the use of UTF-8
   when transmitting JSON text.  However, the vast majority of JSON-
   based software implementations have chosen to use the UTF-8 encoding,
   to the extent that it is the only encoding that achieves
   interoperability.

   Implementations MUST NOT add a byte order mark (U+FEFF) to the
   beginning of a networked-transmitted JSON text.  In the interests of
   interoperability, implementations that parse JSON texts MAY ignore
   the presence of a byte order mark rather than treating it as an
   error.

plokhotnyuk avatar Nov 29 '17 07:11 plokhotnyuk

If you're going to support this, better to support passing an arbitrary Charset.

shawjef3 avatar Mar 12 '18 14:03 shawjef3

I would much prefer support for just String as input/output. Many APIs deal with Strings for JSON, and by only having binary input/output, an extra UTF-8 conversion is required every single time.

EDIT: Just noticed #20

nilskp avatar May 04 '18 18:05 nilskp