runner icon indicating copy to clipboard operation
runner copied to clipboard

GitHub runner cannot send `fetch` with `node`, failing with IPv6 DNS error `UND_ERR_CONNECT_TIMEOUT`

Open undergroundwires opened this issue 1 year ago • 1 comments

Describe the bug Sending more than one (sometimes two or three) requests using fetch results in DNS errors.

To Reproduce I created a repository to reproduce this in as minimal way as possible: https://github.com/undergroundwires/node-fetch-ipv6

I run fetch on using vite test runner on node and without any test runner. I get same results. I realize that the first two requests are successful, I start getting the error after the second or third one.

See the test: test file

Second test (multiple fetch in order) fails, and all run fine locally. I run tests on default GitHub runners using both latest node 20 and 18. I get same errors:

Expected behavior The requests are being sent fine, this was the case before.

Runner Version and Platform

The job runs on ubuntu-latest, see the repository linked in To Reproduce section above.

What's not working?

I get UND_ERR_CONNECT_TIMEOUT errors which is related to IPv6 issues in node. Related issue for node: nodejs/node#41625, Related issue for node fetch: nodejs/undic#1531 Related issue for GitHub runners not being able to do IPv6: actions/runner-images#668, actions/runner#3138 image

Job Log Output

RUN  v1.4.0 /home/runner/work/node-fetch-ipv6/node-fetch-ipv6

stdout | vite-test.spec.ts > fetch in order
Fetching:  https://web.archive.org/web/20221029145712/https://kb.mozillazine.org/Downloads.rdf

stdout | vite-test.spec.ts > fetch in order
Fetching:  https://web.archive.org/web/20240120213614/https://techcommunity.microsoft.com/t5/windows-it-pro-blog/group-configuration-search-highlights-in-windows/ba-p/3263989

 ❯ vite-test.spec.ts  (2 tests | 1 failed) 22219ms
   ❯ vite-test.spec.ts > fetch in order
     → fetch failed

⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  vite-test.spec.ts > fetch in order
TypeError: fetch failed
 ❯ vite-test.spec.ts:26:22
     24|   for (const url of urls) {
     25|     console.log('Fetching: ', url);
     26|     const response = await fetch(url, { method: 'HEAD'});
       |                      ^

Error: TypeError: fetch failed
 ❯ vite-test.spec.ts:26:22

Caused by: ConnectTimeoutError: Connect Timeout Error
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { code: 'UND_ERR_CONNECT_TIMEOUT' }

     27|     await sleep(10000);
     28|     expect(response.status).to.equal(200);

Caused by: Caused by: ConnectTimeoutError: Connect Timeout Error
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { code: 'UND_ERR_CONNECT_TIMEOUT' }
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

 Test Files  1 failed (1)
      Tests  1 failed | 1 passed (2)
   Start at  21:21:50
   Duration  22.54s (transform 42ms, setup 0ms, collect 26ms, tests 22.22s, environment 0ms, prepare 72ms)

undergroundwires avatar Mar 20 '24 10:03 undergroundwires

Here's my workaround (open-source and documented) that I hope that can help you too:

After days of research and trial/error, this is how I got this working:

  1. Create a script called force-ipv4.sh, that configures system to prefer IPv4 over IPv6, call it to configure the machine. It was not easy to find a reliable cross-platform solution and I went Cloudflare WARP for DNS resolution along with some system configurations.
  2. To easily use the script in GitHub workflows, create GitHub action called force-ipv4 and call it in GitHub runners.
  3. Fixes the IPv6 request issues, and you can happily run e.g. fetch API from Node.

Related commit introducing this fix: https://github.com/undergroundwires/privacy.sexy/commit/52fadcd6177ed06216be9c67dad57192ae02a4f9

undergroundwires avatar Mar 29 '24 12:03 undergroundwires