undici icon indicating copy to clipboard operation
undici copied to clipboard

Support incorrect deflate content encoding that browser is supporting

Open heroboy opened this issue 2 years ago • 8 comments

Look at the code.

  1. Use the deflateRawSync is not conform to the http standard. But most browser can correctly open it.
  2. The error AboutError is not clearly.
  3. I have a question. Why undici.request not support deflate encoding?
const zlib = require('zlib');
const http = require('http');
const undici = require('undici');
let server = http.createServer((req, res) =>
{
	console.log('request');
	res.writeHead(200, { 'Content-Type': 'text/plain', 'content-encoding': 'deflate' });
	res.end(zlib.deflateRawSync('abc'));
	//res.end(zlib.deflateSync('abc'));
});
server.listen(8989, '0.0.0.0', async function ()
{
	try
	{
		let resp = await undici.fetch('http://127.0.0.1:8989');
		let text = await resp.text();
		console.log(text);
	}
	catch (e)
	{
                //AbortError: The operation was aborted
		console.log('request error');
		console.log(e);
	}
});

heroboy avatar Sep 03 '21 09:09 heroboy

PR welcome

ronag avatar Sep 03 '21 10:09 ronag

The AbortError seems like a bug. Should probably be another error. So this issue is actually 2 different problems.

ronag avatar Sep 03 '21 10:09 ronag

chrome code that inflecting the data:

https://source.chromium.org/chromium/chromium/src/+/main:net/filter/gzip_source_stream.cc;l=134

heroboy avatar Sep 03 '21 11:09 heroboy

I'm -1 on this one. https://datatracker.ietf.org/doc/html/rfc7230#section-4.2.2

The "deflate" coding is a "zlib" data format [...] Note: Some non-conformant implementations send the "deflate" compressed data without the zlib wrapper.

szmarczak avatar Sep 03 '21 13:09 szmarczak

But can we just add a flag, just like the strictContentLength.

heroboy avatar Sep 04 '21 06:09 heroboy

@mcollina wdyt? node-fetch has support for this https://github.com/node-fetch/node-fetch/blob/8b54ab4509b541dadcd2d66aa5cf0995ad54ac15/src/index.js#L258-L270

ronag avatar Sep 13 '21 06:09 ronag

I think those http servers are still around, so we should unfortunately support it.

I would mark this as "good first issue", so if somebody needs this they know they can implement it.

mcollina avatar Sep 13 '21 06:09 mcollina