brianmario
Yajl fails to decode null characters in strings
Yajl fails to decode null characters in json-encoded strings, silently discarding them.
Yajl::Parser.parse("\"\\u0000\"")
results in "" when it should be "\000".
JSON gem gets this right, for example (array added because JSON gem doesn't like parsing strings on their own):
>> JSON.parse("[\"\\u0000\"]")
=> ["\000"]
Whatever else is in the string, the null bytes are just discarded:
>> Yajl::Parser.parse("\"foo\\u0000bar \\u0000\"")
=> "foobar "
>> JSON.parse("[\"foo\\u0000bar \\u0000\"]")
=> ["foo\000bar \000"]
Dammit my comment was lost in the transfer...
Anyway I had a few examples in here of how Chrome, Safari and Firefox handle this. Chrome and Safari act like Yajl whereas Firefox acts like the JSON gem.
Also, the output from nodejs:
> JSON.parse("[\"foo\\u0000bar \\u0000\"]")
[ 'foo\u0000bar \u0000' ]
Will research more and get back to you.
Chrome is bad at displaying strings containing the null byte (it displays them as if it wasn't there). It certainly doesn't strip them when it parses JSON:
JSON.parse(""\u0000"") //-> "" [the null byte is invisible in chrome] JSON.parse(""\u0000"") == "\u0000" //-> true JSON.parse(""\u0000"") == "" //-> false [actually an empty string]
I've sent a patch to YAJL to fix this https://github.com/lloyd/yajl/pull/19 — would you like me to create a pull request for here too? (I'm not sure how you go about keeping the copy of yajl you have in sync with upstream)
saw that pull request was merged. thanks very much for the fix, Conrad. I hope it makes it into yajl-ruby soon.