cURL fails in the latest mac 12.7.4 and 14.4.1 runner releases

[mmastrac]

Description

Hi all,

For some strange reason, our mac arm64 CI started failing as soon as the runners ticked over to 14.4.1.

It appears that cURL is unable to connect to a localhost or local unix socket for a server that is certainly running at the time.

Example failed run: https://github.com/denoland/deno/actions/runs/8544295631/job/23410919147?pr=23212

Actual output from cURL:

[tls_test 012.13] error: Error: curl https://localhost:50061/d/1 failed: 7:
[tls_test 012.13]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[tls_test 012.13]                                  Dload  Upload   Total   Spent    Left  Speed
[tls_test 012.13] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
[tls_test 012.13] curl: (7) Failed to connect to localhost port 50061 after 0 ms: Couldn't connect to server
[tls_test 012.13] 
[tls_test 012.13]     throw new Error(
[tls_test 012.13]           ^
[tls_test 012.13]     at curl (file:///Users/runner/work/deno/deno/tests/unit/tls_test.ts:1066:11)
[tls_test 012.13]     at eventLoopTick (ext:core/01_core.js:168:7)
[tls_test 012.13]     at async curlFakeHttpsServer (file:///Users/runner/work/deno/deno/tests/unit/tls_test.ts:1078:18)
[tls_test 012.13] 

Build info:

Current runner version: '2.314.1'
Operating System
  macOS
  14.4.1
  23E224
Runner Image
  Image: macos-14-arm64
  Version: 20240402.2
  Included Software: https://github.com/actions/runner-images/blob/macos-14-arm64/20240402.2/images/macos/macos-14-arm64-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/macos-14-arm64%2F20240402.2
Runner Image Provisioner
  2.0.361.1+33a726b54233caea13315dba3887b0e7c5a1d482

Platforms affected

• [ ] Azure DevOps
• [X] GitHub Actions - Standard Runners
• [ ] GitHub Actions - Larger Runners

Runner images affected

• [ ] Ubuntu 20.04
• [ ] Ubuntu 22.04
• [ ] macOS 11
• [ ] macOS 12
• [ ] macOS 13
• [ ] macOS 13 Arm64
• [ ] macOS 14
• [X] macOS 14 Arm64
• [ ] Windows Server 2019
• [ ] Windows Server 2022

Image version and build link

20240402.2

https://github.com/denoland/deno/actions/runs/8544295631/job/23410919147?pr=23212

Is it regression?

Yes

Expected behavior

cURL should connect successfully (as it does on other platforms)

Actual behavior

cURL fails to connect with cURL error 7 (failed to connect)

Repro steps

1. Perform a cURL against a server listening on a localhost port.
2. cURL fails to connect.

[bartlomieju]

FWIW the same problem happened on macOS12 too.

[mmastrac]

Example of failing macOS 12 build below. Note that we upgraded this runner to macOS-13 as a temporary fix which worked, but macOS-14 started failing today when those images updated.

https://github.com/denoland/deno/actions/runs/8529068224/job/23364054712

Current runner version: '2.314.1'
Operating System
  macOS
  12.7.4
  21H1123
Runner Image
  Image: macos-12
  Version: 20240329.1
  Included Software: https://github.com/actions/runner-images/blob/macOS-12/20240329.1/images/macos/macos-12-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/macOS-12%2F20240329.1
Runner Image Provisioner
GITHUB_TOKEN Permissions

Same error:

[tls_test 011.72] error: Error: curl https://localhost:3550/d/1 failed: 7:
[tls_test 011.72]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[tls_test 011.72]                                  Dload  Upload   Total   Spent    Left  Speed
[tls_test 011.72] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
[tls_test 011.72] curl: (7) Failed to connect to localhost port 3550 after 0 ms: Couldn't connect to server
[tls_test 011.72] 
[tls_test 011.72]     throw new Error(
[tls_test 011.72]           ^
[tls_test 011.72]     at curl (file:///Users/runner/work/deno/deno/tests/unit/tls_test.ts:1078:11)
[tls_test 011.72]     at eventLoopTick (ext:core/01_core.js:168:7)
[tls_test 011.72]     at async curlFakeHttpsServer (file:///Users/runner/work/deno/deno/tests/unit/tls_test.ts:1091:18)
[tls_test 011.72] 

[mmastrac]

Might be related to https://github.com/actions/runner-images/pull/9586

[mmastrac]

@erik-bershel Unfortunately this appears to be caused by https://github.com/actions/runner-images/pull/9586.

It looks like we're preferentially binding ipv6, so curl can no longer connect to our localhost address with the --ipv4 flag. We might rethink how we bind local addresses and prefer ipv4, but as a heads up the change definitely had some unintended side-effects.

[erik-bershel]

Hey @mmastrac! I'll reconsider those changes again shortly. Until then, I suggest you to use the workaround. Namely, delete or clear the config file. As you did as I see. 😆

[mmastrac]

Thanks @erik-bershel! Appreciate it 😀

[erik-bershel]

Hey there! New macOS images delivered. Closing this item as completed.