AppImage for ARM Linux

[SomberNight]

At the moment we only build AppImages for x86_64 Linux. See https://github.com/spesmilo/electrum/tree/master/contrib/build-linux/appimage

It would be nice to be able to do the same for ARM (maybe armhf?).

Ideally, the build script would NOT assume the host to be ARM, so e.g. it would cross-compile or similar.

[bauerj]

There are Docker images available that can run ARM-based operating systems on an amd64 CPU. See e.g. https://www.balena.io/docs/reference/base-images/base-images/#building-arm-containers-on-x86-machines

[SomberNight]

Nice! I've been looking at https://ownyourbits.com/2018/06/27/running-and-building-arm-docker-containers-in-x86/

[SomberNight]

I did a bit of research on this. To avoid redoing it next time I look at this, some random useful links:

• "Build script for Qt5 in debug mode with SIP bindings and PyQt5" https://gist.github.com/elpaso/8ca33ddca336b7ecf0c25c77c7367572
• "Installing PyQt5" https://www.riverbankcomputing.com/static/Docs/PyQt5/installation.html
• "Running and building ARM Docker containers in x86" https://ownyourbits.com/2018/06/27/running-and-building-arm-docker-containers-in-x86/
• "How to Build and Run ARM Docker Images on x86 Hosts" https://matchboxdorry.gitbooks.io/matchboxblog/content/blogs/build_and_run_arm_images.html
• Qt docs https://doc.qt.io/qt-5/build-sources.html https://doc.qt.io/qt-5/linux-requirements.html https://wiki.qt.io/Building_Qt_5_from_Git https://doc.qt.io/qt-5/configure-options.html https://bugreports.qt.io/browse/QTBUG-41200
• "Install PyQt5 on Raspberry for Python3.6" https://stackoverflow.com/questions/55062050/install-pyqt5-on-raspberry-for-python3-6
• random Qt compilation failure I ran into https://github.com/Intel-Media-SDK/MediaSDK/issues/34

[robster7674]

I gave this a try on my PinePhone using Manjaro/PhoSh beta 23. Result was an error from Docker:

💬 INFO:  building docker image.
Sending build context to Docker daemon  24.06kB
Step 1/13 : FROM ubuntu:18.04@sha256:9bc830af2bef73276515a29aa896eedfa7bdf4bdbc5c1063b4c457a4bbb8cd79
docker.io/library/ubuntu:18.04@sha256:9bc830af2bef73276515a29aa896eedfa7bdf4bdbc5c1063b4c457a4bbb8cd79: Pulling from library/ubuntu
4028d4a2ab03: Pull complete 
Digest: sha256:9bc830af2bef73276515a29aa896eedfa7bdf4bdbc5c1063b4c457a4bbb8cd79
Status: Downloaded newer image for ubuntu:18.04@sha256:9bc830af2bef73276515a29aa896eedfa7bdf4bdbc5c1063b4c457a4bbb8cd79
 ---> d7efed8348f7
Step 2/13 : ENV LC_ALL=C.UTF-8 LANG=C.UTF-8
 ---> Running in bb2dd5bfa82e
Removing intermediate container bb2dd5bfa82e
 ---> 7623cde525b0
Step 3/13 : ENV DEBIAN_FRONTEND=noninteractive
 ---> Running in 6308eee79232
Removing intermediate container 6308eee79232
 ---> 3fa6fe2d054f
Step 4/13 : RUN apt-get update -q &&     apt-get install -qy         git=1:2.17.1-1ubuntu0.9         wget=1.19.4-1ubuntu2.2         make=4.1-9.1ubuntu1         autotools-dev=20180224.1         autoconf=2.69-11         libtool=2.4.6-2         autopoint=0.19.8.1-6ubuntu0.3         xz-utils=5.2.2-1.3         libssl-dev=1.1.1-1ubuntu2.1~18.04.15         libssl1.1=1.1.1-1ubuntu2.1~18.04.15         openssl=1.1.1-1ubuntu2.1~18.04.15         zlib1g-dev=1:1.2.11.dfsg-0ubuntu2         libffi-dev=3.2.1-8         libncurses5-dev=6.1-1ubuntu1.18.04         libncurses5=6.1-1ubuntu1.18.04         libtinfo-dev=6.1-1ubuntu1.18.04         libtinfo5=6.1-1ubuntu1.18.04         libsqlite3-dev=3.22.0-1ubuntu0.4         libusb-1.0-0-dev=2:1.0.21-2         libudev-dev=237-3ubuntu10.53         libudev1=237-3ubuntu10.53         gettext=0.19.8.1-6ubuntu0.3         libzbar0=0.10+doc-10.1build2          libdbus-1-3=1.12.2-1ubuntu1.2         libxkbcommon0=0.8.0-1ubuntu0.1         libxkbcommon-x11-0=0.8.0-1ubuntu0.1         libxcb1=1.13-1         libxcb-xinerama0=1.13-1         libxcb-randr0=1.13-1         libxcb-render0=1.13-1         libxcb-shm0=1.13-1         libxcb-shape0=1.13-1         libxcb-sync1=1.13-1         libxcb-xfixes0=1.13-1         libxcb-xkb1=1.13-1         libxcb-icccm4=0.4.1-1ubuntu1         libxcb-image0=0.4.0-1build1         libxcb-keysyms1=0.4.0-1         libxcb-util1=0.4.0-0ubuntu3         libxcb-render-util0=0.3.9-1         libx11-xcb1=2:1.6.4-3ubuntu0.4         libc6-dev=2.27-3ubuntu1.5         libc6=2.27-3ubuntu1.5         libc-dev-bin=2.27-3ubuntu1.5         &&     rm -rf /var/lib/apt/lists/* &&     apt-get autoremove -y &&     apt-get clean
 ---> Running in a6b4f2fff62c
failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: bpf_prog_query(BPF_CGROUP_DEVICE) failed: function not implemented: unknown

A similar error was mentioned recently on https://discuss.getsol.us/d/8202-docker-not-working-related-to-recent-update/3. Is this perhaps due to kernel config? Currently running:

[manjaro@manjaro-arm appimage]$ uname -r
5.16.16-1-MANJARO-ARM

It does indeed seem that BPF_CGROUP_DEVICE is not enabled:

[manjaro@manjaro-arm appimage]$ zgrep BPF /proc/config.gz 
CONFIG_BPF=y
CONFIG_HAVE_EBPF_JIT=y
CONFIG_ARCH_WANT_DEFAULT_BPF_JIT=y
# BPF subsystem
# CONFIG_BPF_SYSCALL is not set
CONFIG_BPF_JIT=y
CONFIG_BPF_JIT_DEFAULT_ON=y
# end of BPF subsystem
# CONFIG_NETFILTER_XT_MATCH_BPF is not set
# CONFIG_BPFILTER is not set
# CONFIG_TEST_BPF is not set

Note: imo we should have the image build on non-Docker installations as well. Since the project ships a binary, not a container image, requiring people to run Docker just for this purpose seems counter-intuitive.

[robster7674]

Researching further shows that it's CONFIG_BPF_SYSCALL that must be enabled, not (just) CONFIG_CGROUP_BPF, see https://github.com/opencontainers/runc/issues/2959#issuecomment-988464142. I guess we'll need to file a Manjaro issue on this to get the kernel config changed.

[robster7674]

Filed https://github.com/manjaro-pinephone/phosh/issues/35. Note that it's CONFIG_CGROUP_BPF that should also be enabled, not BPF_CGROUP_DEVICE.

[SomberNight]

Note: imo we should have the image build on non-Docker installations as well. Since the project ships a binary, not a container image, requiring people to run Docker just for this purpose seems counter-intuitive.

I think there is a misunderstanding here. Docker is used to build the binary. This dockerfile is not for the enduser to run Electrum on their machine. It is there to set up a container that can be used to build the binary. Prior discussion in this thread is about how to change these build scripts so that they can be used to "cross-compile" the binary - i.e. build a binary on a physical amd64 machine that can be run by endusers on arm64 machines.

Another approach could be to adapt the build scripts so that at least they can also be used on arm64 machines and build arm64 binaries there (so no cross-compiling). -- I would discourage this approach as the project maintainers only have amd64 machines and would prefer to build all binaries on such machines.

Also, if you are trying to run the current build scripts on arm, note that the ubuntu base image is hash-pinned, and I guess the hash might be architecture-dependent: https://github.com/spesmilo/electrum/blob/250795f137281b8d2c8d29081949ee781a1c0b2a/contrib/build-linux/appimage/Dockerfile#L4

[robster7674]

@SomberNight : thanks for the background information. No, there is no misunderstanding: I am aware Docker is used for building and not needed for running. Still, I feel that on a non-Docker machine, the binary should be able to build as well.

Your comment on cross-compilation makes sense. If it's possible to cross-compile for arm64 on amd64, isn't it possible to do the other way around as well? Let me dig into that.

If hardware is an issue, I am happy to contribute to get some arm64 hardware, like a Raspberry Pi or a PinePhone, to the developers so tests can be run natively on that hardware.

[robster7674]

Looks like "sudo docker run --platform linux/amd64 ubuntu" could work on arm64. However, I won't know until I get the 5.17 kernel running on my beta23 PinePhone, see https://github.com/manjaro-pinephone/phosh/issues/35#issuecomment-1088367719.

[robster7674]

Got a bit further now that I have 5.17 kernel. I ran "sudo pacman-mirrors --api --set-branch unstable" to get 5.17.1-2:

[manjaro@manjaro-arm appimage]$ uname -r
5.17.1-2-MANJARO-ARM

I re-ran ./build.sh, and got an error about pyqt5:

ERROR: Could not find a version that satisfies the requirement PyQt5==5.15.6 (from versions: none)
ERROR: No matching distribution found for PyQt5==5.15.6

Then I installed python-pyqt5, and re-ran ./build.sh. Same error, apparently because the version I get is not exactly the same:

[manjaro@manjaro-arm appimage]$ yay -Q python-pyqt5
python-pyqt5 5.15.6-7

5.15.6-7 as opposed to the required 5.15.6. Provided I can figure out where to change the requirement, should I try?

[SomberNight]

PyQt5 is installed here: https://github.com/spesmilo/electrum/blob/ad41f4aed01a8848ce9139f22f84a433c3b7c3cc/contrib/build-linux/appimage/make_appimage.sh#L123-L124 this references contrib/deterministic-build/requirements-binaries.txt: https://github.com/spesmilo/electrum/blob/ad41f4aed01a8848ce9139f22f84a433c3b7c3cc/contrib/deterministic-build/requirements-binaries.txt#L79 (which is committed to the repo to pin hashes; it is generated from: https://github.com/spesmilo/electrum/blob/ad41f4aed01a8848ce9139f22f84a433c3b7c3cc/contrib/requirements/requirements-binaries.txt)

So it is using pip, which downloads packages from PyPI. There are no binary releases for arm linux, see: https://pypi.org/project/PyQt5/5.15.6/#files There is a source release (tar.gz) though. I think because of the --no-binary :all: --only-binary PyQt5,PyQt5-Qt5,cryptography flags, pip will not attempt to get the tar.gz and compile PyQt5 from source -- I would expect that to fail btw.

The easy workaround is to install PyQt5 from the system package manager (note that this is all running inside a docker container, so I am not talking about your host system). You can add it around here: https://github.com/spesmilo/electrum/blob/ad41f4aed01a8848ce9139f22f84a433c3b7c3cc/contrib/build-linux/appimage/Dockerfile#L10 (you would also need to remove pyqt5 from contrib/deterministic-build/requirements-binaries.txt)

[SomberNight]

https://github.com/bitcoin-core/HWI/pull/615 is interesting and simple, but it still leaves the question of what to do with pyqt.

[kiminuo]

Is https://github.com/bitcoin-core/HWI/pull/691 possibly relevant for you?

[SomberNight]

Only to the degree that pyside6 has pre-built wheels for manylinux aarch64. There is discussion re switching to Qt6 in https://github.com/spesmilo/electrum/issues/8007 Though note that atm we are using pyqt, not pyside. pyqt6 does not have pre-built wheels for manylinux aarch64.

Perhaps we should build Qt ourselves, just like we do it for the Android build. Then again, that might blow up the build times, just like we have seen for Android.