🐞BUG: not working with 'chunked' data! (fix inside)

[k1tzu]

test with http://emojitracker.com/ It sends 'chunked' data with headers: "transfer-encoding":"chunked" To get this working we need three things:

1. catch the 'transfer-encoding' === 'chunked'
2. don't .push to resData but send to user
3. don't close the socket but write to it

You need to add code to catch chunked data : from

...
            res.on("data", function (chunk) {
                    resData.push(chunk);
}
...

to

...
            res.on("data", function (chunk) {
                if (resHeader['transfer-encoding'] === 'chunked') {
                    processResponse([chunk], {'chunked': true});
                } else {
                    resData.push(chunk);
                }
}
...

Then you need to move whole async code

...
                var serverResData;
                async.series([

                    //ungzip server res
                    function (callback) {
...

to the function

            function processResponse(resData, chunked) {
                var serverResData;
                async.series([

                    //ungzip server res
                    function (callback) {
...

notice it takes resData as an argument!

and replace the code there for response not to close the socket

                        //send response
                    }, function (callback) {
                        if (global._throttle) {
                            console.log(21);
                            var thrStream = new Stream();

                            var readable = thrStream.pipe(global._throttle.throttle());
                            readable.pipe(userRes);

                            thrStream.emit("data", serverResData);
                            thrStream.emit("end");
                            callback();
                        } else {
                             userRes.end(serverResData);
                            callback();
                        }

to

                        //send response
                    }, function (callback) {
                        if (global._throttle) {
                            console.log(21);
                            var thrStream = new Stream();

                            var readable = thrStream.pipe(global._throttle.throttle());
                            readable.pipe(userRes);

                            thrStream.emit("data", serverResData);
                            thrStream.emit("end");
                            callback();
                        } else {
                            if (chunked) {
                                userRes.write(serverResData);
                            } else {
                                userRes.end(serverResData);
                            }
                            callback();
                        }

[codingfishman]

@k1tzu Chunked data is useful especaily when dealing with large data, and we haven't get a "good" way for this yet.

AnyProxy offers Programable Rule to support changing the behavior of Request and Response, and we may need a whole data in the rule, so we can change it in a complete way. The side effect is we swallow the chunked data to a complete one.

We need to balance, maybe a switch option, to support the chunked data and programing the response at the same time.

[zachsa]

I'm trying to cache an image that is a response from a GET request:

const beforeSendResponse = async (req, res) => {
  const { method } = req.requestOptions
  if (method.toUpperCase() === 'GET') {
    cacheResponse(req, res)
  }
}

But I'm finding that I can't access the body. Is this related? I'm finding I can't access the 'data' event on the res param. Should I be able to? Sorry if this question doesn't make much sense! I'm looking at the req, and res objects than I have prior.