Added Record Separator (ASCII RS/0x1E) as a valid top level record separator in multiple content mode

[yurikus]

Added Record Separator (ASCII RS/0x1E) as a valid top level record separator in multiple content mode. We use ASCII RS as record separator in our json record stream.

See: https://github.com/JamesNK/Newtonsoft.Json/issues/1355 https://github.com/JamesNK/Newtonsoft.Json/commit/6aed848eca0bb7ee8612495c203dc44a71e0f12a

[yurikus]

@JamesNK Could you merge this please? A fairly simple change, which would helps us out a lot! Thanks!

[JamesNK]

I'm not sure about this. There are 3 other ASCII separators characters. Why not them as well? Why not semi-colons or fullstops? etc

No delimiter makes sense to me, and comma makes sense because it is already a separator in JSON, but a line needs to drawn somewhere.

I think you should either change your separator character, or copy the source code and introduce that change yourself.

[yurikus]

TLDR: We have to use reflection to increment _charPos ....

Hi, thank you for quick response. ASCII defines 0x1e as Record Separator character, which was chosen to delimit individual records in the stream of multiple tens of megabytes of data. I'm not at the liberty to change the character. My benchmarking showed 40% perf gain when using JsonTextReader which is patched via reflection to skip the character vs reading the stream and then examining data. In my opinion, using this character is justified. I've tried building a custom version, which led me to binding redirect hell, as other packages depend on the publicKeyToken of real dll.

[yurikus]

@JamesNK Hi James, what's the verdict?