support binary data via overrideMimeType() (or other means of not assuming the returned data is utf8)

[asutherland]

https://developer.mozilla.org/En/XMLHttpRequest/Using_XMLHttpRequest#Receiving_binary_data_in_older_browsers documents using:

req.overrideMimeType('text/plain; charset=x-user-defined');

as a way to force web browsers to not treat the returned data as utf-8. This is useful if you are doing something like trying to fetch an image in web browsers and also would like that same code to work in node.

Currently, this library forces the data stream to be interpreted as utf-8 which is potentially lossy, and definitely lossy in the case of png images. Specifically, the first byte 0x89 when round-tripped through utf-8 gets converted into the replacement character 0xffd and comes back (utf-8 encoded) as 0xef 0xbf 0xbd.

[driverdan]

Thanks for finding this. There are 2 missing features in node-XHR here. It's missing overrideMimeType and it's missing proper encoding detection.

[pwnall]

overrideMimeType is mentioned as a hack. I would suggest implementing responseType = "arraybuffer" instead (MDN reference), because node.js supports typed arrays.

[gabrielhurley]

+1 on adding support for binary transfers, FWIW. I just ran into this and ended up just hacking around it usng node's http.request directly since there's no way to change XMLHttpRequest's forced utf8 encoding.

A cursory look through the code looks like the two places that need changes are on the call to setEncoding and on the write call, where it needs the second argument for the encoding.

[pwnall]

If it helps, this is the code I used in my XHR node.js implementation.

https://github.com/pwnall/node-xhr2/blob/master/src/xml_http_request_upload.coffee#L20 https://github.com/pwnall/node-xhr2/blob/master/src/xml_http_request_upload.coffee#L83

[nkzawa]

Is this still an issue? Can I send Buffer or ArrayBuffer?

[gaurav21r]

Any Updates on this? I came from here

http://stackoverflow.com/questions/29276660/unable-to-download-file-from-google-drive-using-api-node-js

I see someone from Dropbox also having this issue. The above question is regarding downloading files from Google Drive. I guess any REST API we connect to from Node.JS will always have a requirement of downloading binary content via XHR.