bun icon indicating copy to clipboard operation
bun copied to clipboard

`fetch` cant handle `gzip` compressed data

Open Kapsonfire-DE opened this issue 3 years ago • 5 comments

Version

0.1.4

Platform

Microsoft Windows NT 10.0.25151.0 x64

What steps will reproduce the bug?

async function getInfo(rId) {
    var res = await fetch(`https://users.roblox.com/v1/users/${rId}`).catch(err => console.log(err));

    var data = res.data;

    return data;
}

var rId = 520887847;
console.log(getInfo(rId));

How often does it reproduce? Is there a required condition?

Always

What is the expected behavior?

Getting the json response I guess :-D

What do you see instead?

Zlib error error: fetch() failed – ZlibError url: "https://users.roblox.com/v1/users/520887847"

Additional information

Tested by discord users: Magic (338164255549423620)

Ubuntu WSL: Linux momtamer69 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Powershell: Microsoft Windows NT 10.0.22000.0 x64

Kapsonfire (233904020509556736)

Debian WSL: Linux KAPS-PC 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 GNU/Linux
Microsoft Windows NT 10.0.25151.0 x64

Kapsonfire-DE avatar Jul 13 '22 18:07 Kapsonfire-DE

What happens if you remove Accept-Encoding? Bun handles compression automatically, but it's possible this breaks if Accept-Encoding is explicitly passed as explicitly passing it lacks test coverage

Jarred-Sumner avatar Jul 13 '22 19:07 Jarred-Sumner

you are not allowed to change Accept-Encoding in fetch api

sorry didnt remove it from my code on top. changing headers for accept-encoding does not work. even if you drop it, it doesnt work. it was just a test to evade this error

//edit: removed it from code

Kapsonfire-DE avatar Jul 13 '22 19:07 Kapsonfire-DE

Tested, I can reproduced it on:

Darwin mac 21.5.0 Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:29 PDT 2022; root:xnu-8020.121.3~4/RELEASE_ARM64_T8101 arm64

hisamafahri avatar Jul 14 '22 05:07 hisamafahri

na its not about deflate @Jarred-Sumner its gzip the server sends

Kapsonfire-DE avatar Jul 14 '22 13:07 Kapsonfire-DE

fwiw i was able to get a request working from the same zlib error by adding Accept-Encoding: gzip to the headers

const getTreeRes = await fetch(
  `https://endpoint.here`,
  {
    method: 'GET',
    headers: {
      'Content-Type': 'application/json',
      'Accept-Encoding': 'gzip',
    },
  },
)

malonehedges avatar Aug 06 '22 04:08 malonehedges

We need to fix this issue, and will. A temporary workaround is to provide an Accept-Encoding header.

const response = await fetch("https://example.com/", {
  headers: {
    "Accept-Encoding": "identity",
  }
});

Electroid avatar Nov 02 '22 22:11 Electroid

Should be fixed in #1510

Seems like a common HTTP server library is just broken when deflate is preferred over gzip

Jarred-Sumner avatar Nov 15 '22 06:11 Jarred-Sumner