superagent icon indicating copy to clipboard operation
superagent copied to clipboard

can superAgent return Buffer data ?

Open lessfish opened this issue 9 years ago • 10 comments

I love using superAgent to set http request, but the charset of my requested page is gbk. Now I learn to use iconv-lite to decode, but it needs Buffer.

so I wonder if superAgent can return Buffer data ? and How ?

with great thanks !

lessfish avatar Jan 25 '16 08:01 lessfish

This middleware might help you: https://github.com/magicdawn/superagent-charset

timneutkens avatar Feb 11 '16 19:02 timneutkens

For the case described above, the superagent-charset middleware is a great solution. However, receiving binary data instead of (utf-8 decoded) strings is a very generic feature. See issues #825 and #824.

I'd love to have a method like this

req.binary(true)

that would cause res.body to be the binary response. The type of this value would be a Buffer for Node.js and a Uint8Array, Blob or ArrayBuffer in the browser.

maarten-t avatar Feb 21 '16 14:02 maarten-t

We've added Blob support on the client #888

kornelski avatar Feb 21 '16 14:02 kornelski

I didn't know about the undocumented responseType('blob') feature from #888.

The value 'blob' wouldn't make sense for Node, though, since it doesn't have (native) support for Blobs. Wouldn't a more generic and versatile API be to use req.responseType to override the HTTP Content-Type header of the response? Then, the argument would have to be a mime-type.

With such an API you can force binary output with responseType('application/octet-stream') (or responseType('binary') for short). Calling responseType('application/json') would force parsing JSON even when the server is not sending the right Content-Type.

It would even make it possible to force a certain character encoding, for instance by calling responseType('text/html; charset=UTF-8'). This feature would be useful because the response character encoding is often omitted, this feature would be useful. Supporting the GBK encoding (see the start of this thread) and many others would require an extra library though.

maarten-t avatar Feb 22 '16 09:02 maarten-t

I don't see it as related to content type at all. You could ask for text/plain in a Buffer if you wanted bytes instead of a string. And on client-side you may want image/png as Blob or ArrayBuffer — and there's nothing PNG-specific in an ArrayBuffer.

kornelski avatar Feb 22 '16 14:02 kornelski

I'm receiving an xlsx spreadsheet which I want to pass as a buffer to js-xlsx. The following works in parse:

(NB wrapped in new Promise((resolve)=>...rest of request...) )

.parse((res)=>{
  let buffer = [];
  res.on('data', (chunk)=>{
    buffer.push(chunk);
  });
  res.on('end', ()=>
    resolve(Buffer.concat(buffer)));
  });

shaunc avatar Mar 13 '17 14:03 shaunc

To get binary data in Node:

request.get(…)
    .buffer(true).parse(superagent.parse.image)
    .then(res => res.body)

kornelski avatar Mar 13 '17 18:03 kornelski

Nice @kornelski ... I thought that superagent.parse.image must have something to do with images... :)

shaunc avatar Mar 14 '17 00:03 shaunc

In the latest version you can use superagent.parse['application/octet-stream'], but .image is shorter to write, and both do the same thing :)

kornelski avatar Mar 14 '17 14:03 kornelski

To get binary data in Node:

request.get(…)
    .buffer(true).parse(superagent.parse.image)
    .then(res => res.body)

How would it be using async await?

let img = await superagent.get(data.url).buffer(true).parse(superagent.parse.image);

How do I get ".then (res => res.body)"?

dwitmexico avatar Jun 26 '19 17:06 dwitmexico