How to decrypt a GZIP response?

[maximseshuk]

Hey! Thanks for the great proxy server. The question arose as to the best way to do the decryption GZIP response? Thanks!

[gr3p1p3]

You could use native nodejs to do this, but I think I will extend the data buffers with some special method to easier parse headers & body!! This will maybe be a next feature :)

If you need this already now... then look here:

https://nodejs.org/docs/latest/api/zlib.html#zlib

[fopinappb]

It'd be really nice to have this covered by the proxy itself as noone can inspect encoded responses without decoding them :)

For anyone bumping into this issue, sharing a bit of code I used. It assumes http response was stored in a object and headers split from body, so this.responseBody only refers to: httpResponse.split("\r\n\r\n")[1] (pseudo)

This handles chunked transfer-encoding (merging all chunks) and then decodes

	private mergeBodyChunks(): Buffer {
		// merge all chunks in single response body
		// it should be done in a "streamy" way but we already have the full response in memory
		let modifiedBody = this.responseBody;
		if (
			this.getResponseHeaderValue("transfer-encoding") !== undefined &&
			this.getResponseHeaderValue("transfer-encoding").toLowerCase() == "chunked"
		) {
			const chunkSeparator = Buffer.from(
				this.responseBody.indexOf("\r\n") >>> 0 < this.responseBody.indexOf("\n") >>> 0 ? "\r\n" : "\n"
			);
			const chunks: Buffer[] = [];
			const final = Buffer.concat([Buffer.from("0"), chunkSeparator]);
			while (modifiedBody.indexOf(final) > 0) {
				const split = modifiedBody.indexOf(chunkSeparator);
				if (split < 0) throw new HttpRequestResponseParsingError("chunk separator not found");
				const chunkLengthStr = modifiedBody.subarray(0, split).toString();
				const chunkLength = parseInt(chunkLengthStr, 16);
				if (isNaN(chunkLength)) throw new HttpRequestResponseParsingError(`invalid chunk length: ${chunkLengthStr}`);
				const chunk = modifiedBody.subarray(split + chunkSeparator.length, split + chunkSeparator.length + chunkLength);
				if (chunk.length != chunkLength)
					throw new HttpRequestResponseParsingError(
						`invalid chunk: read ${chunk.length} bytes but expected ${chunkLength}`
					);
				chunks.push(chunk);
				modifiedBody = modifiedBody.subarray(split + chunkSeparator.length + chunkLength + chunkSeparator.length);
			}
			// if terminating chunk is missing, something went wrong
			if (modifiedBody.indexOf(final) < 0)
				throw new HttpRequestResponseParsingError("missing 0-sized chunk (terminator): " + this.responseBody.toString("base64"));
			modifiedBody = Buffer.concat(chunks);
		}
		return modifiedBody;
	}

	private decodeBody() {
		/*
		Available compressions: gzip, deflate, compress, br
		They can be chained **in the order in which they were applied**
		  Content-Encoding: deflate, gzip

		src: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding#syntax
		*/

		const encodingHeader = this.getResponseHeaderValue("content-encoding");
		if (encodingHeader == undefined) return;

		let decodedData: Buffer;
		try {
			decodedData = this.mergeBodyChunks();
		} catch (e) {
			if (e instanceof HttpRequestResponseParsingError) {
				logger.error(`failed to merge chunked response: ${e}`);
				return;
			} else {
				throw e;
			}
		}

		// code based on axios https://github.com/axios/axios/blob/bdf493cf8b84eb3e3440e72d5725ba0f138e0451/lib/adapters/http.js#L383
		// but supporting multiple encodings
		let decoder: (buf: zlib.InputType) => Buffer;
		const decoded = encodingHeader
			.split(",")
			.reverse()
			.every((encoding) => {
				decoder = undefined;
				encoding = encoding.trim();
				switch (encoding) {
					case "gzip":
					case "compress":
					case "deflate":
						decoder = zlib.unzipSync;
						break;
					case "br":
						decoder = zlib.brotliDecompressSync;
						break;
				}
				if (decoder == undefined) {
					logger.error(`no decoder for "${encoding}"`);
					return false;
				}
				try {
					decodedData = decoder(decodedData);
				} catch (e) {
					logger.error(`failed to decode "${encoding}" - ${e}`);
					return false;
				}
				return true;
			});

		if (decoded) {
			this.responseBody = decodedData;
			this.responseBodyDecoded = decoded;
		}
	}

Hope it's useful for others

[gr3p1p3]

Try with the version @1.11.7

const server = new ProxyServer({
    verbose: true,
    intercept: true,
    injectResponse: (data, session) => {
        if (session.response.complete //if response is finished
            && session.response.headers['content-encoding'] === 'gzip') { //body is gzip
            const zlib = require('zlib');
            zlib.gunzip(session.rawResponse, function (err, decoded) {
                console.log('decoded response', decoded?.toString())
            });
        }
        return data;
    }
});