fetch() fails on "https://jamsocket.xyz" (doesn't fail on node/deno/curl)

[pretentious7]

trafficstars

Version

0.1.13

Platform

Linux 5.15.0-47-generic #51-Ubuntu SMP Thu Aug 11 07:51:15 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

What steps will reproduce the bug?

(async () => {
try {
  console.log(await fetch("https://jamsocket.xyz"))
} catch(e) {
  console.error(e)
}
})()

export {}

(wrapped the async in a function for easier verification in node)

How often does it reproduce? Is there a required condition?

every time

What is the expected behavior?

Response (<<NOT SURE>> KB) {
  ok: true,
  url: "https://jamsocket.xyz/",
  statusText: "OK",
  redirected: false,
  bodyUsed: true,
  status: 200,
  Blob (<<NOT SURE>> KB)
}

What do you see instead?

ConnectionClosed: fetch() failed
 path: "https://jamsocket.xyz"

Additional information

No response

[Jarred-Sumner]

There is something strange happening with TLS and I don't understand what it is yet

It's definitely something to do with TLS 1.3.

[Jarred-Sumner]

Maybe it's tickets

[jdjkelly]

Seems like it's definitely TLS 1.3 - another reproducible host where is happening: https://api.flexpa.com/config

try {
  await fetch('https://api.flexpa.com/config').then((response) => {
    if (!response.ok) {
      throw new Error(`HTTP error! Status: ${response.status}`);
    }

    return response.blob();
  })
} catch(e) {
  console.log(e)
}

Unexpectedly returns

ConnectionClosed: fetch() failed
 path: "https://api.flexpa.com/config"

[klaascuvelier]

I am facing the same issue with the Notion api. I noticed the need repro label, @Jarred-Sumner; what exactly would you like to see (given the example from the OP is pretty straight forward?)

[0x4a61636f62]

At least on version 0.1.11 fetch was working flawlessly. I'm not sure when it happened, but on version 0.1.13 one of my working requests is now throwing ConnectionClosed: fetch() failed.

[Jarred-Sumner]

@jastraberg what specific website? It could be an issue with the BoringSSL handshake process which has different code now. If you can link to the specific website / repro, I can debug this

The broader TLS 1.3 issue needs to be fixed but I'm not really sure how yet.

[0x4a61636f62]

@Jarred-Sumner I have made a private repo https://github.com/jastraberg /repro-1263 and added you as a collaborator.

• All of the relevant code is found in the directory: ./src/modules/api/classes/rest-client.class.ts
• The entry point is main.ts in the root folder and can be run with bun run paper .
• The endpoints/website are found in ./src/modules/api/constants/endpoints.constant.ts.

[renhiyama]

Any updates on this? Or any temporary workaround?

[blackfyre]

This problem has my bun project suspended... so any updates/workarounds/... would be very welcome.

[Jarred-Sumner]

I have a fix shortly.

[Jarred-Sumner]

When moving to uSockets for the http client, I forgot to call SSL_set_tlsext_host_name and uSockets apparently doesn't do that. I'm sad this took 18 days to fix

[renhiyama]

Welcome to failure management! (It's a meme, not roasting you ofc, just making you less stressful Jarred!)

[cyberalien]

Experiencing the same error with 0.3.0 when trying to fetch https://unicode.org/Public/emoji/15.0/emoji-test.txt

[Electroid]

@robobun

await fetch("https://unicode.org/Public/emoji/15.0/emoji-test.txt");

[robobun]

@Electroid here you go!

ConnectionClosed: fetch() failed
 path: "https://unicode.org/Public/emoji/15.0/emoji-test.txt"

Code

await fetch("https://unicode.org/Public/emoji/15.0/emoji-test.txt");

Ran using the latest build of Bun, an all-in-one JavaScript runtime.

[Jarred-Sumner]

Spent some time investigating this

The behavior we're seeing is:

1. We send a request like this:

GET /Public/emoji/15.0/emoji-test.txt
	accept-encoding: identity
	Connection: keep-alive
	User-Agent: Bun/0.4.1_debug
	Accept: */*
	Host: unicode.org

2. It returns a Response like this:

< 	Date: Tue, 17 Jan 2023 01:19:25 GMT
< 	Server: Apache/2.4.38 (Debian)
< 	Content-Security-Policy: upgrade-insecure-requests;
< 	Last-Modified: Tue, 16 Aug 2022 16:34:20 GMT
< 	ETag: "90d58-5e65e5205627e"
< 	Accept-Ranges: bytes
< 	Content-Length: 593240
< 	Vary: Accept-Encoding
< 	Connection: close
< 	Content-Type: text/plain; charset=utf-8

3. It includes the first 300-400 bytes, then the server closes the connection.

What I'm wondering here is: is it possible that uSockets is closing the socket before reading the data?

[robobun]

@Jarred-Sumner here you go!

error: Expected identifier after "Date:" in namespaced JSX name
         Tue, 17 Jan 2023 01:19:25 GMT
/tmp/bun-
/tmp/bun-XB5i4K/index.js:1:9 8

Code

< 	Date: Tue, 17 Jan 2023 01:19:25 GMT
< 	Server: Apache/2.4.38 (Debian)
< 	Content-Security-Policy: upgrade-insecure-requests;
< 	Last-Modified: Tue, 16 Aug 2022 16:34:20 GMT
< 	ETag: "90d58-5e65e5205627e"
< 	Accept-Ranges: bytes
< 	Content-Length: 593240
< 	Vary: Accept-Encoding
< 	Connection: close
< 	Content-Type: text/plain; charset=utf-8

Ran using the latest build of Bun, an all-in-one JavaScript runtime.

[Jarred-Sumner]

Fixed between Bun v0.5.9 and Bun v0.6.2

bun on  main
❯ cat /tmp/abc.ts
───────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: /tmp/abc.ts
───────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ await fetch("https://unicode.org/Public/emoji/15.0/emoji-test.txt");
───────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────

bun on  main
❯ bun-0.5.9 /tmp/abc.ts
ConnectionClosed: The socket connection was closed unexpectedly. For more information, pass `verbose: true` in the second argument to fetch()
 path: "https://unicode.org/Public/emoji/15.0/emoji-test.txt"

bun on  main
❯ bun-0.6.2 /tmp/abc.ts