reliable
reliable copied to clipboard
michelle
Binary data gets garbaged
When trying to send an image of just some KB, the content gets converted to a sequence of escaped Unicode glyphs ("u1234") getting a transfered size sometimes more than double bigger, and worst than that, message gets truncated because of the extra size. For text files (for example HTML) there's not this problem.
I actually cannot reproduce this. Are you converting your image to a string first?
Yes, I'm moving JSON strings. El 03/05/2013 06:47, "Michelle Bu" [email protected] escribió:
I actually cannot reproduce this. Are you converting your image to a string first?
— Reply to this email directly or view it on GitHubhttps://github.com/michellebu/reliable/issues/11#issuecomment-17378465 .
Why not just directly send JSON objects with Blobs? Binarypack takes care of everything and will allow you to serialize binary. I'm guessing this is your own thing.
Michelle
On Thu, May 2, 2013 at 10:58 PM, Jesús Leganés Combarro < [email protected]> wrote:
Yes, I'm moving JSON strings. El 03/05/2013 06:47, "Michelle Bu" [email protected] escribió:
I actually cannot reproduce this. Are you converting your image to a string first?
— Reply to this email directly or view it on GitHub< https://github.com/michellebu/reliable/issues/11#issuecomment-17378465> .
— Reply to this email directly or view it on GitHubhttps://github.com/michellebu/reliable/issues/11#issuecomment-17379628 .
It's probably good to just build in utf8 as an option for reliable, because runtime really takes a hit in serializing utf8.
Maybe it's my fault :-) I'm using JSON on my project since begin 9 months ago, mainly because it's available everywhere natively and don't need to add any extra library to use it, but I have pending to check other serialization schemes that could be smaller and faster. The fact of using BinaryPack on reliable library it's just and implementation detail for me and should be transparent, since I'm not using it directly. El 03/05/2013 08:00, "Michelle Bu" [email protected] escribió:
Why not just directly send JSON objects with Blobs? Binarypack takes care of everything and will allow you to serialize binary. I'm guessing this is your own thing.
Michelle
On Thu, May 2, 2013 at 10:58 PM, Jesús Leganés Combarro < [email protected]> wrote:
Yes, I'm moving JSON strings. El 03/05/2013 06:47, "Michelle Bu" [email protected] escribió:
I actually cannot reproduce this. Are you converting your image to a string first?
— Reply to this email directly or view it on GitHub< https://github.com/michellebu/reliable/issues/11#issuecomment-17378465> .
— Reply to this email directly or view it on GitHub< https://github.com/michellebu/reliable/issues/11#issuecomment-17379628> .
— Reply to this email directly or view it on GitHubhttps://github.com/michellebu/reliable/issues/11#issuecomment-17379674 .
Maybe it's the reason why it's so slow for me? El 03/05/2013 08:01, "Michelle Bu" [email protected] escribió:
It's probably good to just build in utf8 as an option for reliable, because runtime really takes a hit in serializing utf8.
— Reply to this email directly or view it on GitHubhttps://github.com/michellebu/reliable/issues/11#issuecomment-17379703 .
http://kentonv.github.io/capnproto/
At last found it :-P I didn't remember the name, but I was thinking about using Cap'n Proto as serialization protocol, or something custom derived from it. Main advantage is that's memory mapped using offsets, so I could transfer the data using arrayBuffer as binaryType and on receiver just use ArrayViewBuffer to access to its content without conversion. What do you think? El 03/05/2013 08:11, "[email protected]" [email protected] escribió:
Maybe it's the reason why it's so slow for me? El 03/05/2013 08:01, "Michelle Bu" [email protected] escribió:
It's probably good to just build in utf8 as an option for reliable, because runtime really takes a hit in serializing utf8.
— Reply to this email directly or view it on GitHubhttps://github.com/michellebu/reliable/issues/11#issuecomment-17379703 .
Cap'n Proto was designed with static languages in mind and IMO has little place in Javascript if we're not involving any other lower level languages. The biggest issue I see with it is that it requires a schema--for something like PeerJS it would introduce unnecessary overhead. BinaryPack is actually a modified version of MessagePack, which is pretty widely adopted as a web serialization format.
MessagePack and its improved cousin BinaryPack seems to be better candidates... :-D It only keep to see if there's any implementation using ArrayBuffers :-)
2013/5/3 Michelle Bu [email protected]
Cap'n Proto was designed with static languages in mind and IMO has little place in Javascript if we're not involving any other lower level languages. The biggest issue I see with it is that it requires a schema--for something like PeerJS it would introduce unnecessary overhead. BinaryPack is actually a modified version of MessagePack, which is pretty widely adopted as a web serialization format.
— Reply to this email directly or view it on GitHubhttps://github.com/michellebu/reliable/issues/11#issuecomment-17416094 .
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un monton de sitios diferentes, simplemente escribe un sistema operativo Unix." – Linus Tordvals, creador del sistema operativo Linux
Yeah basically there's no need to convert your file blob into a binary string using binary serialization formats, but it's definitely worthwhile to build in the utf8 options, especially since using utf8 on PeerJS with reliable on Chrome will currently not actually serialize utf8.
On Fri, May 3, 2013 at 4:02 PM, Jesús Leganés Combarro < [email protected]> wrote:
MessagePack and its improved cousin BinaryPack seems to be better candidates... :-D It only keep to see if there's any implementation using ArrayBuffers :-)
2013/5/3 Michelle Bu [email protected]
Cap'n Proto was designed with static languages in mind and IMO has little place in Javascript if we're not involving any other lower level languages. The biggest issue I see with it is that it requires a schema--for something like PeerJS it would introduce unnecessary overhead. BinaryPack is actually a modified version of MessagePack, which is pretty widely adopted as a web serialization format.
— Reply to this email directly or view it on GitHub< https://github.com/michellebu/reliable/issues/11#issuecomment-17416094> .
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un monton de sitios diferentes, simplemente escribe un sistema operativo Unix." – Linus Tordvals, creador del sistema operativo Linux
— Reply to this email directly or view it on GitHubhttps://github.com/michellebu/reliable/issues/11#issuecomment-17422823 .