java icon indicating copy to clipboard operation
java copied to clipboard

Adding encoding as parameter to methods deserialize and serialize

Open leocampos opened this issue 5 years ago • 5 comments

This pull request tries to address the issue https://github.com/json-iterator/java/issues/229

Especially with so many managed services on the Cloud, some of which we do not have any control over the initialization of the JVM, it is important to be able to control the encoding used by the library. I faced recently the problem of having one of these managed services in US-ASCII, but our Strings were encoded with UTF-8. All special characters were transformed to "?".

leocampos avatar Nov 16 '19 19:11 leocampos

@taowen is there any chance this could get merged? it's fixing a pretty big issue

TheNeuralBit avatar Feb 10 '20 19:02 TheNeuralBit

BTW, we are mostly starting to migrate to another JSON parser in which the development is more active.

leocampos avatar Feb 13 '20 09:02 leocampos

@leocampos what's the name of the new JSON parser that you mentioned the development is active

pradeepcheers avatar Aug 06 '21 15:08 pradeepcheers

@pradeepcheers We are currently using Jackson. https://github.com/FasterXML/jackson.

The approach we took, though, was to wrap it around a class of ours, so we can later choose the library and change it in a single place.

leocampos avatar Aug 06 '21 17:08 leocampos

For UTF-16 this PR crashes with:

com.jsoniter.spi.JsonException: read: premature end, head: 1, peek: �, buf: �� { " n a m e " : " T h o m a s  M � l l e r " }
	at com.jsoniter/com.jsoniter.JsonIterator.reportError(JsonIterator.java:143)
	at com.jsoniter/com.jsoniter.JsonIterator.readAny(JsonIterator.java:272)
	at com.jsoniter/com.jsoniter.JsonIterator.deserialize(JsonIterator.java:522)
	at com.jsoniter/com.jsoniter.JsonIterator.deserialize(JsonIterator.java:505)
	at com.jsoniter/com.jsoniter.TestString.test_encoding_different_than_default(TestString.java:41)

Also the test does not work for Java 17 modules as reflective access to Charset.defaultCharset is prohibited.

Brixomatic avatar Aug 16 '22 22:08 Brixomatic