[fix] Not Support identity;q=1, *;q=0

[743v45]

Describe the bug

Node.js version: v22.11.0

OS version: MacOS 15.3.1

koa-compress version: 5.1.1

Description: Accept-Encoding: identity;q=1, *;q=0 not work

Actual behavior

response with Content-Encoding: gzip

Expected behavior

Content-Encoding: identity

Code to reproduce

const Koa = require('koa');
const compress = require('koa-compress');

const app = new Koa();

app.use(compress(
  {
    filter: (contentType) => {
      return true;
    },
    threshold: 1,
    br: false,
  },
));

app.use(async (ctx) => {
    ctx.body = 'This is a sample response that will be compressed if the client supports it.';
});

const port = 3000;
app.listen(port, () => {
    console.log(`Server is running on port ${port}`);
});

Checklist

• [x] I have searched through GitHub issues for similar issues.
• [x] I have completely read through the README and documentation.
• [x] I have tested my code with the latest version of Node.js and this package and confirmed it is still not working.

[jonathanong]

returning a weight of 0 means you still are requesting gzip. what you actually want to request is just Accept-Encoding: identity

[743v45]

@jonathanong

The Chrome browser initiated this request with Accept-Encoding: identity;q=1, *;q=0. This is not my delusion, but koa-compress forced a gzip return.

identity is also a function

identity Indicates the identity function (that is, without modification or compression). This value is always considered as acceptable, even if omitted.

I did not find an authoritative statement that is a perfect match. The following is the description of the gpt-4o

[jonathanong]

yes, the authoritative statement says that it's a priority, not a rejection. content negotiation is a negotiation. we can return identity here, but we don't have to - it does not break specifications because you still requested gzip even if with priority = 0.

we can make this an option, but realistically, it's not practical (browsers and clients generally do not request encodings like this) and most people would prefer the response size savings.

however, IIRC your PR was incorrect as it explicitly rejected p=0, which is incorrect. if the client wants identity, then it should only request identity.

[743v45]

Let me describe the scenario I encountered.

I added ctx.compress = true; to the route <host>/path/to/:filename to support compression. At first, only images were handled, and later videos were adapted.

However, when there is a video request, the request carries Accept-Encoding: identity;q=1, *;q=0, but the response is encoded, which leads to net::ERR_CONTENT_DECODING_FAILED.

Accept-Encoding is a parameter on the client side. From the client's perspective, for Accept-Encoding: identity;q=1, *;q=0, it is best to return identity.

Of course, this is not suitable for the default koa-compress. The filter only filters text types supported by compressible and is not adapted to video/mp4 by default.

It seems that when Chrome initiates a request with Accept-Encoding: identity;q=1, *;q=0, it only wants the identity encoding and doesn't want gzip.

[mempf]

I agree with @743v45 this is a real bug and is causing issues with Chrome and an HLS server I wrote that uses koa-compress.

I have written a more detailed response with a link to the related RFC here: https://github.com/koajs/compress/pull/233#issuecomment-3536191507

I believe this bug should be re-opened.

[uhop]

How to repro it without programming *;q=0? What is the actual real-world use case?

[mempf]

How to repro it without programming *;q=0? What is the actual real-world use case?

The original steps to reproduce should work with sending the Accept-Encoding: identity;q=1, *;q=0 directly.

In terms of real world use case this comes up with Chrome 141+ requesting HLS playlists and video segments with this header and expecting uncompressed data.

This is a behaviour you should be able to see if you go to this HLS test page and look at the network requests being sent by Chrome. You must ensure that the override native flag is disabled as it only seems to be Chromes native HLS support that sends headers like this. https://videojs-http-streaming.netlify.app/?debug=false&autoplay=false&muted=false&fluid=false&minified=false&sync-workers=false&liveui=true&llhls=true&url=https%3A%2F%2Fd2zihajmogu5jn.cloudfront.net%2Fhls-webvtt%2Fmaster.m3u8&type=application%2Fx-mpegurl&keysystems=&buffer-water=false&exact-manifest-timings=false&pixel-diff-selector=false&network-info=true&dts-offset=false&override-native=false&object-fit=false&use-mms=true&preload=auto&mirror-source=true&forced-subtitles=false&native-text-tracks=false

[jonathanong]

@mempf thanks for a real-world use-case. It seems like HLS playlists are compressible? Not sure why they would request it uncompressed. PR welcomed

[mempf]

@mempf thanks for a real-world use-case. It seems like HLS playlists are compressible? Not sure why they would request it uncompressed. PR welcomed

They seem to be doing some byte range stuff with their requests I imagine wouldn't work properly if compressed. I'm not thrilled with it either as yeah the playlists tend to compress well. Regardless it exposes a bug.

[mempf]

Can we re-open and merge https://github.com/koajs/compress/pull/233 from @743v45 ?

[uhop]

@mempf I am prepping a new release and it will include the fix for this bug as well. Could you prepare a test for your use case? It'll help a lot. See https://github.com/koajs/compress/tree/master/tests how they are done.

[mempf]

@mempf I am prepping a new release and it will include the fix for this bug as well. Could you prepare a test for your use case? It'll help a lot. See https://github.com/koajs/compress/tree/master/tests how they are done.

Will do. I'll keep an eye out for a new release and try it with our hls-server.