Support AArch64

[jonathanmetzman]

Now that it is supported on GCE.

[jonathanmetzman]

Initial work on this was started in https://github.com/google/oss-fuzz/pull/4591/files

[guidovranken]

This is awesome. A lot of cryptography fuzzers will benefit from this.

[guidovranken]

I know this is WIP but I would like to prepare some projects for AArch64 support.

When I do

infra/helper.py build_fuzzers nettle --architecture=aarch64

I get

Successfully built b7165bc337dc
Successfully tagged gcr.io/oss-fuzz/nettle:latest
Keeping existing build artifacts as-is (if any).
Running: docker run --rm --privileged -i --cap-add SYS_PTRACE -e FUZZING_ENGINE=libfuzzer -e SANITIZER=address -e ARCHITECTURE=aarch64 -e FUZZING_LANGUAGE=c++ -v /mnt/2tb/oss-fuzz-aarch64/oss-fuzz/build/out/nettle:/out -v /mnt/2tb/oss-fuzz-aarch64/oss-fuzz/build/work/nettle:/work -t gcr.io/oss-fuzz/nettle
---------------------------------------------------------------
Compiling libFuzzer to /usr/lib/libFuzzingEngine.a... cp: cannot stat '/usr/local/lib/clang/*/lib/linux/libclang_rt.fuzzer-aarch64.a': No such file or directory
Building fuzzers failed.

I've tried both the master and arm branches and pulling the images again.

[jonathanmetzman]

I know this is WIP but I would like to prepare some projects for AArch64 support.

I'd warn you against doing this. There is a significant blocker that could prevent Aarch64 support landing for a while: Lack of support for ARM on Google Cloud Build. My old cross-compilation method no longer works and I'm not sure I want to go with this approach anymore since:

1. I basically cargo-culted the implementation before and I can't figure out how to do it again, and there isn't anything I can copy.
2. I don't want to bloat our builders so much since it will hurt CIFuzz/CFL users.

So at this point I'm considering building using QEMU instead of cross compilation.

When I do

infra/helper.py build_fuzzers nettle --architecture=aarch64

I get

Successfully built b7165bc337dc
Successfully tagged gcr.io/oss-fuzz/nettle:latest
Keeping existing build artifacts as-is (if any).
Running: docker run --rm --privileged -i --cap-add SYS_PTRACE -e FUZZING_ENGINE=libfuzzer -e SANITIZER=address -e ARCHITECTURE=aarch64 -e FUZZING_LANGUAGE=c++ -v /mnt/2tb/oss-fuzz-aarch64/oss-fuzz/build/out/nettle:/out -v /mnt/2tb/oss-fuzz-aarch64/oss-fuzz/build/work/nettle:/work -t gcr.io/oss-fuzz/nettle
---------------------------------------------------------------
Compiling libFuzzer to /usr/lib/libFuzzingEngine.a... cp: cannot stat '/usr/local/lib/clang/*/lib/linux/libclang_rt.fuzzer-aarch64.a': No such file or directory
Building fuzzers failed.

I've tried both the master and arm branches and pulling the images again.

Here is a rough guide if you want to ignore the advice above.

docker run --privileged linuxkit/binfmt:v0.8
docker buildx create --name buildxbuilder
docker buildx use buildxbuilder
docker buildx build --progress plain -t gcr.io/oss-fuzz/skcms --file $HOME/oss-fuzz/projects/skcms --platform linux/arm64 --load

[jonathanmetzman]

I don't know if I'm going to add this QEMU support to helper.py either btw.

[evverx]

I'm not sure if it was rolled out or not but judging by https://oss-fuzz-build-logs.storage.googleapis.com/log-640610d8-c435-4843-9585-7605ae64bb9c.txt systemd appears to have failed to build on aarch64 with

Starting Step #43 - "compile-libfuzzer-address-aarch64"
Step #43 - "compile-libfuzzer-address-aarch64": Already have image (with digest): gcr.io/cloud-builders/docker
Step #43 - "compile-libfuzzer-address-aarch64": ---------------------------------------------------------------
Step #43 - "compile-libfuzzer-address-aarch64": Compiling libFuzzer to /usr/lib/libFuzzingEngine.a... cp: cannot stat '/usr/local/lib/clang/*/lib/linux/libclang_rt.fuzzer-aarch64.a': No such file or directory
Step #43 - "compile-libfuzzer-address-aarch64": ********************************************************************************
Step #43 - "compile-libfuzzer-address-aarch64": Failed to build.
Step #43 - "compile-libfuzzer-address-aarch64": To reproduce, run:
Step #43 - "compile-libfuzzer-address-aarch64": python infra/helper.py build_image systemd
Step #43 - "compile-libfuzzer-address-aarch64": python infra/helper.py build_fuzzers --sanitizer address --engine libfuzzer --architecture aarch64 systemd
Step #43 - "compile-libfuzzer-address-aarch64": ********************************************************************************
Finished Step #43 - "compile-libfuzzer-address-aarch64"
ERROR
ERROR: build step 43 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1

[jonathanmetzman]

Oh damn, sorry about that.

[jonathanmetzman]

I'll disable these projects.

[evverx]

@jonathanmetzman thanks! I didn't know aarch64 was turned on there. Generally systemd along with its fuzz targets is built regularly on all sorts of architectures by the CI so it should be safe to bring it back once the underlying infrastructure is ready (assuming the fuzz targets are built "natively" without having to add a meson cross-file).

[jonathanmetzman]

clusterfuzz-images/base build fails on ARM with this error:

==============================================================================
Starting installation of google-fluentd
==============================================================================

Installing agent for Debian or Ubuntu.
Warning: apt-key output should not be parsed (stdout is not a terminal)
OK
Get:1 http://packages.cloud.google.com/apt google-cloud-logging-focal InRelease [5484 B]
Get:2 http://packages.cloud.google.com/apt google-cloud-logging-focal/main arm64 Packages [615 B]
Fetched 6099 B in 0s (17.5 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Package google-fluentd is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'google-fluentd' has no installation candidate
dpkg-query: no packages found matching google-fluentd
google-fluentd: unrecognized service

==============================================================================
Errors occurred while installing google-fluentd-. See the log snippet
above or run:
  sudo service google-fluentd status
==============================================================================
sed: can't read /etc/google-fluentd/google-fluentd.conf: No such file or directory
The command '/bin/sh -c curl -sSO https://dl.google.com/cloudagents/install-logging-agent.sh &&     bash install-logging-agent.sh || true &&     sed -i 's/flush_interval 5s/flush_interval 60s/' /etc/google-fluentd/google-fluentd.conf' returned a non-zero code: 2