nix-installer icon indicating copy to clipboard operation
nix-installer copied to clipboard

Spurious failure on GitHub arm64 linux runners

Open lovesegfault opened this issue 11 months ago • 6 comments

Error

While trying out the new arm64 runners, I saw this spurious failure:

Error: 
   0: Install failure
   1: Error executing action
   2: Action `create_upstream_init_service` errored
   3: Action `configure_init_service` errored
   4: Failed to execute command `"systemctl" "daemon-reload"`
      stdout: 
      stderr: Reload daemon failed: Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms)

   4: exited with status code: 1

https://github.com/lovesegfault/nix-config/actions/runs/12869379921/job/35878063511

Metadata

key value
version 0.34.0
os linux
arch aarch64

lovesegfault avatar Jan 20 '25 13:01 lovesegfault

Another case of it:

https://github.com/lovesegfault/nix-config/actions/runs/12871819768/job/35885950927?pr=5023

lovesegfault avatar Jan 20 '25 17:01 lovesegfault

I'm seeing this more and more, for whatever reason it's quite common on these hosts.

lovesegfault avatar Jan 21 '25 14:01 lovesegfault

"Great."

I wonder if this is just an issue with the image / configuration / whatever on these machines (since they are fairly new). They also appear to publish 22.04 images -- could you try with that and see if that has the same issue?

cole-h avatar Jan 21 '25 15:01 cole-h

This is super annoying. We've actually been using these machines for four months now: https://github.com/DeterminateSystems/nix-installer/commit/ab99f41e5a1afd1da0f71e49e575d76f6489a742#diff-f6d710dfea0b18b6ee14c3c05f43cf6057236e294f4be992dbb52249786fa88a

I suspect this is due to our machines using 22.04 instead of 24.04, which wasn't available at the time. ~I'll upgrade~ I have upgraded our machines to 24.04 so we feel the pain :)

grahamc avatar Jan 21 '25 15:01 grahamc

I moved my repo back to 22.04 for now, let's see if it's stable or if the issue reoccurs :)

lovesegfault avatar Jan 21 '25 15:01 lovesegfault

I'm seeing varying failures of compile commands on Linux-aarch64 builds of PyMuPDF (https://github.com/pymupdf/PyMuPDF).

The builds are done with docker/setup-qemu-action@v1 on ubuntu-latest.

They worked fine on 2025-01-17, but failed the next time we did a build, on 2025-02-06.

Three very different failures, so it does look like a problem with the system:

1:

2025-02-06T13:12:56.8549581Z   (+2m07.1s): -b: m: __main__.py:1591:build: (+0.8s): -b: m: __main__.py:1591:build: cc: internal compiler error: Segmentation fault signal terminated program cc1
2025-02-06T13:12:56.8629437Z   (+2m07.1s): -b: m: __main__.py:1591:build: (+0.8s): -b: m: __main__.py:1591:build: Please submit a full bug report,
2025-02-06T13:12:56.8736628Z   (+2m07.1s): -b: m: __main__.py:1591:build: (+0.8s): -b: m: __main__.py:1591:build: with preprocessed source if appropriate.
2025-02-06T13:12:56.8806392Z   (+2m07.1s): -b: m: __main__.py:1591:build: (+0.8s): -b: m: __main__.py:1591:build: See <http://bugzilla.redhat.com/bugzilla> for instructions.
2025-02-06T13:12:56.8882927Z   (+2m07.2s): -b: m: __main__.py:1591:build: (+0.8s): -b: m: __main__.py:1591:build: make: *** [build/PyMuPDF-aarch64-shared-tesseract-bsymbolic-Py_LIMITED_API_0x03090000-release/source/fitz/output-pcl.o] Error 4
2025-02-06T13:12:56.8977372Z   (+2m07.2s): -b: m: __main__.py:1591:build: (+0.8s): -b: m: __main__.py:1591:build: make: *** Waiting for unfinished jobs....
2025-02-06T13:12:59.1676788Z   (+2m09.4s): -b: m: __main__.py:1591:build: (+0.8s): -b: m: __main__.py:1591:build:
2025-02-06T13:12:59.1700724Z   (+2m09.4s): -b: m: __main__.py:1591:build: [returned e=2]

2:

2025-02-06T13:24:23.5882274Z   (+36.2s): -b: m: __main__.py:1591:build: (+0.9s): -b: m: __main__.py:1591:build: /bin/sh: line 1:   803 Segmentation fault      (core dumped) cc -ffunction-sections -fdata-sections -pipe -O2 -DNDEBUG -DTOFU_CJK_EXT -Iinclude -MMD -MP -o build/PyMuPDF-aarch64-shared-tesseract-bsymbolic-Py_LIMITED_API_0x03090000-release/source/fitz/draw-affine.o -c source/fitz/draw-affine.c -Wall -Wsign-compare -Wdeclaration-after-statement -fPIC -Ithirdparty/freetype/include -Iscripts/freetype -Ithirdparty/gumbo-parser/src -Ithirdparty/harfbuzz/src -Ithirdparty/libjpeg -Iscripts/libjpeg -Ithirdparty/lcms2/include -DHAVE_LCMS2MT -Ithirdparty/mujs -Ithirdparty/zlib -Ithirdparty/jbig2dec -Ithirdparty/openjpeg/src/lib/openjp2 -DOPJ_STATIC -DOPJ_HAVE_INTTYPES_H -DOPJ_HAVE_STDINT_H -DHAVE_LEPTONICA -Ithirdparty/leptonica/src -DHAVE_TESSERACT -Ithirdparty/tesseract/include -Iscripts/tesseract -Ithirdparty/extract/include -Ithirdparty/extract/src
2025-02-06T13:24:23.6090223Z   (+36.2s): -b: m: __main__.py:1591:build: (+0.9s): -b: m: __main__.py:1591:build: make: *** [build/PyMuPDF-aarch64-shared-tesseract-bsymbolic-Py_LIMITED_API_0x03090000-release/source/fitz/draw-affine.o] Error 139

3:

2025-02-10T21:03:22.0813957Z   (+4m31.9s): -b: m: __main__.py:1591:build: (+0.8s): -b: m: __main__.py:1591:build: /bin/sh: line 1:  3077 Segmentation fault      (core dumped) cc -ffunction-sections -fdata-sections -pipe -O2 -DNDEBUG -DTOFU_CJK_EXT -Iinclude -MMD -MP -o build/PyMuPDF-aarch64-shared-tesseract-bsymbolic-Py_LIMITED_API_0x03090000-release/source/pdf/pdf-run.o -c source/pdf/pdf-run.c -Wall -Wsign-compare -Wdeclaration-after-statement -fPIC -Ithirdparty/freetype/include -Iscripts/freetype -Ithirdparty/gumbo-parser/src -Ithirdparty/harfbuzz/src -Ithirdparty/libjpeg -Iscripts/libjpeg -Ithirdparty/lcms2/include -DHAVE_LCMS2MT -Ithirdparty/mujs -Ithirdparty/zlib -Ithirdparty/jbig2dec -Ithirdparty/openjpeg/src/lib/openjp2 -DOPJ_STATIC -DOPJ_HAVE_INTTYPES_H -DOPJ_HAVE_STDINT_H -DHAVE_LEPTONICA -Ithirdparty/leptonica/src -DHAVE_TESSERACT -Ithirdparty/tesseract/include -Iscripts/tesseract -Ithirdparty/extract/include -Ithirdparty/extract/src