io: handling of binary data

[graphicore]

In metapolator we are currently implementing zipping of UFO data, mainly for user downloads from the browser. However, since a zip is binary data, writing binary data to disk becomes a topic, now.

from: MetapolatorProject

            var zipped = this.getZippedInstance(masterName, instanceName, precision, 'nodebuffer');
            this._io.writeFile(false, instanceName, zipped);

This is nodejs specific code, because it writes a nodeJS Buffer to io.staticNodeJS, which will handle the buffer object just right. In nodeJS,

• for writing: "The encoding option is ignored if data is a buffer. It defaults to 'utf8'."
• for reading: "If no encoding is specified, then the raw buffer is returned."

The "nodebuffer" argument is inherited from http://stuk.github.io/jszip/documentation/api_jszip/generate.html

Possible values for type :

• base64 (default) : the result will be a string, the binary in a base64 form.
• string : the result will be a string in "binary" form, using 1 byte per char (2 bytes).
• uint8array : the result will be a Uint8Array containing the zip. This requires a compatible browser
• arraybuffer : the result will be a ArrayBuffer containing the zip. This requires a compatible browser.
• blob : the result will be a Blob containing the zip. This requires a compatible browser.
• nodebuffer : the result will be a nodejs Buffer containing the zip. This requires nodejs.

For IO we need a backend-agnostic way to read and write binary data. Maybe each io-api implementation could provide an interface or information to guide io-using code the right way. I think the overview above, from jszip is a good starting point to think about the different available options.

In Browsers we would generally use ArrayBuffer in nodeJS Buffer, I suppose.

Note: the InMemory io implementation would have to change it's binary types between NodeJS and the Browsers. The REST client should ideally handle binary data correct even from the simplest possible HTTP file server (something like python -m SimpleHTTPServer or even the file:// protocoll), at least we need a way to receive Binary data (maybe a readBinaryFile interface)

NOTE: similar, the plistLib has to handle binary data as well. Maybe that needs improvement, too.