autobahn-js icon indicating copy to clipboard operation
autobahn-js copied to clipboard

More docs on choice and use of serialization formats

Open oberstet opened this issue 5 years ago • 0 comments

Crossbar.io supports these serialization formats:

AutobahnJS supports these serialization formats:

  • JSON, MessagePack, CBOR

AutobahnJS does not support UBJSON, but it could - though there is little reason to do so.


We should add more docs rgd the following stuff:

  • [ ] What are the characteristics of the different formats, and what are our recommendations in different scenarios (as usual, there is no single answer .. it depends .. on client/router support and upon what aspect a user wants to optimize for .. CPU cycles / memory usage on client and router, traffic volume, ..)
  • [ ] How does WAMP auto-negotiate serialization format in principle
  • [ ] How does Crossbar.io work in this regard - well, link to the respective docs there
  • [ ] How does AutobahnJS work here: a) when user does nothing specifically, and b) when the user wants to enforce a single serializer (knowing the router has that enabled), or how to prioritize

Rgd the first of above list. Optimize/select serializer. This also depends on:

  • specific data
  • use of WebSocket compression

For example, say you have a lot of "real numbers" to transmit.

Example:

  • JS internally represents any number as a binary double precision floating point in IEEE754 format. 8 bytes.
  • When the real number is actually is used for money values, say with 2 decimals for EUR and 2 decimals for EUR Cent precision, those values will actually are decimal fixed point, and binary floats cannot efficiently (or precisely) represent decimal fractional values - it will blow up the bit representation
  • in this scenario JSON will consume 5 bytes (4 digits and the "dot"), whereas MessagePack/CBOR/UBJSON when using a JS double will lead to 8 bytes
  • to reap above benefits of 5 bytes (vs 8), one has to use a decimal JS library of course on the client side too
  • the picture again changes when you put WebSocket compression into the mix, because there are only 1000 different values possible (4 decimals), and compression will reduce the consumes bytes on the wire to log_2(1000) on average
  • now this goes even deeper, because WebSocket compression has a lot of knobs: eg "context takeover", which controls wether the compression context extends beyond a single WebSocket message or not. when you send a single number per WebSocket message without premessage-deflate context takeover, there won't be any compression

because there are so many variables in this, we should explain the underlying principles (for those who really wanna get it) AND have recommendation for the other 99% users that just want a recommendation that works "most of the time"

in summary: if absolute minimum wire traffic is desired, you have to measure it using specific selection of serializer/transport/compression settings AND a representative set of YOUR data. this is essentially completely unrelated to AutobahnJS, but just due to the fact that there cannot come up with a single compression algorithm that minimizes for any data - that is not possible in principle (one can mathematically prove that).

if you now dig even deeper, you will come out at this stuff (and the related field) https://en.wikipedia.org/wiki/Chaitin%27s_constant

anyways. probably doing too deep now already;)

oberstet avatar Aug 23 '18 04:08 oberstet