CI issues: OSX agents occasionally crashing with `1 leaked semaphore`

[h-vetinari]

trafficstars

I've seen a pretty substantial increase in flakiness of osx agents recently, which fail during what usually seems to be the artefact analysis step (after the main build has run through).

The most memorable (if perhaps irrelevant things) I remembered from those failures that they often mentioned 1 leaked semaphore.

Below is another example that now showed up in arrow CI:

/usr/local/miniconda/etc/profile.d/conda.sh: line 9: 18881 Segmentation fault: 11  "$CONDA_EXE" $_CE_M $_CE_CONDA "$@"
/usr/local/miniconda/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown

Is someone aware of this already and/or know what this could be caused by?

[pitrou]

The "1 leaked semaphore" is probably a red herring, a consequence of the segfault and certainly not its cause.

[traversaro]

I think @Tobias-Fischer reported a similar problem in https://gitter.im/conda-forge/conda-forge.github.io?at=630c403b11a6a83d048a8b88 , and one of the hypothesis was that it is a problem related to lief.

[Tobias-Fischer]

Yeah - we are manually pinning liblief in some feedstocks by adding this to the conda-forge.yml and rerendering:

remote_ci_setup:
   - liblief=0.11.5
   - conda-forge-ci-setup=3

See https://github.com/conda-forge/qt-main-feedstock/pull/43/commits/1c9f19a56d19b4e414dc89811da5e8519d064eff

[h-vetinari]

For those subscribed, this should be mitigated with the next conda-smithy release: https://github.com/conda-forge/conda-smithy/pull/1684

Thanks @mbargull!

[wolfv]

There is a new version of lief that came out today (0.12.3) that might fix a couple of segfaults. I've tried to reproduce segfaults (unsuccesfully): https://github.com/lief-project/LIEF/issues/814

[mbargull]

Thanks for letting us know about the 0.12.3 release. There have been multiple Mach-O-related (segfault) fixes, so this could very well help :).