MacOS: If brew fails to install ccache, skip using ccache?

[jonesmz]

See this CI Run: https://github.com/ninja-build/ninja/runs/5777867654?check_suite_focus=true

Install ccache
  /bin/bash -xc brew install ccache
  + brew install ccache
  ==> Downloading https://ghcr.io/v2/homebrew/core/hiredis/manifests/1.0.2
  ==> Downloading https://ghcr.io/v2/homebrew/core/hiredis/blobs/sha256:14ae7b3adb354b673a3744e9d849d6698846d5162d3d5f0eb8f9d8837c858e75
  ==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:14ae7b3adb354b673a3744e9d849d6698846d5162d3d5f0eb8f9d8837c858e75?se=2022-03-31T20%3A35%3A00Z&sig=%2FbbWXeQuHptzScNdWVeq0RKGBbHa1MTOGnW%2Fic5VZuY%3D&sp=r&spr=https&sr=b&sv=2019-12-12
  curl: (35) error:14004410:SSL routines:CONNECT_CR_SRVR_HELLO:sslv3 alert handshake failure
  Error: ccache: Failed to download resource "hiredis"
  Download failed: https://ghcr.io/v2/homebrew/core/hiredis/blobs/sha256:14ae7b3adb354b673a3744e9d849d6698846d5162d3d5f0eb8f9d8837c858e75
  Error: Restoring cache failed: Error: The process '/bin/bash' failed with exit code 1

[hendrikmuhs]

sounds like a problem with your curl - SE recommends installing curl from brew.

I guess this problem only applies to a certain macos version and the version of curl that ships with it.

regarding your question:

If brew fails to install ccache, skip using ccache?

I don't think so. That's error silencing, I think we should not. If we silence here, users won't notice other problems. The right mitigation if you are not able to fix curl is to take out this action for this platform using a worflow conditional. But as said, I suggest to find the reason for the curl error.

[jonesmz]

I have no idea what the reason for the curl error is. I'm just using your ccache-action as-is. I don't otherwise interact with curl in any way.

[hendrikmuhs]

The action use brew to install ccache, similar to what your PR does. You can e.g. swap the steps in your PR and install the dependencies 1st. If you add ccache there it should fail with the same error.

The SE post I found: https://unix.stackexchange.com/questions/192944/how-to-fix-curl-sslv3-alert-handshake-failure

It recommends installing curl from brew before this action.

Another recommendation for this PR is setting fail-fast to false:

  strategy:
      fail-fast: false

Right now github stops as soon as it errors, but it would actually be nice if your workflow executes the other macos version. it would be good to know if the problem only happens for 10.15.

If that's the case, I could also add a workaround in the action to install curl via brew before ccache for macos 10.15.

[jonesmz]

I'm seeing that the failure is inconsistent. Sometimes (with no changes to the macos.yml) it works, sometimes it doesn.t

This makes me think that there is potentially a misconfigured github supplied macos runner, or the problem is based on network variance.

[jonesmz]

Added fail-fast: false as requested

[hendrikmuhs]

I'm seeing that the failure is inconsistent. Sometimes (with no changes to the macos.yml) it works, sometimes it doesn.t

I guess brew is using different mirrors to download packages, if unlucky it chooses one where you hit the sslv3 problem.

What's your perceived ratio of successful vs. failing workflow runs?

[jonesmz]

Out of about 6-7 runs, it failed twice with ssl errors. But I didn't have enough data points to say there was much of a pattern.

[jonashaag]

How about moving from Homebrew to GH download for macOS? Homebrew is MUCH slower. Maybe also do it for Ubuntu?

[hendrikmuhs]

I don't see binary releases for mac and linux on github, but only windows.

At least on ubuntu apt seems the way to go. brew is the de-facto although not official standard... :thinking:

@jonesmz and maybe other finding this issue: It would be nice to confirm if upgrading curl before installing ccache via the action fixes the problem.

So the options we have:

• retry the install n times (with some probability we get a "good" mirror)
• install it from another place, e.g. github (not good if we have to install from source)
• upgrade curl on mac (2 calls to brew install, to be confirmed, what if that triggers the error in the 1st call)
• do nothing (I can't believe we are the only affected users, either github fixes the base image or brew does not use "broken mirrors")

[jonashaag]

I don't see binary releases for mac and linux on github, but only windows.

You're right, didn't see that!

[jonashaag]

Could it be that this was just a hickup of GH packages/container registry, which Homebrew uses as the download source?

[hendrikmuhs]

Could it be that this was just a hickup of GH packages/container registry, which Homebrew uses as the download source?

That's what I mean with "broken mirrors": homebrew uses several mirrors for download to distribute the load. By chance it picks one that lets the download fail. I agree this could just be a hickup. We should only add a workaround if its a permanent issue.

[jonashaag]

I think Homebrew stopped using mirrors and now uses GHCR only

[arximboldi]

This seems to be failking all the time now...

[hendrikmuhs]

@arximboldi do you have a link to a failing one?

[arximboldi]

I think it was just some downtime of ghcr returning 503. It's working now. Sorry for the noise!