Arm64 build

[mauritslamers]

I would like to run NW.js on a PineBook (http://pine64.org). While the same processor is used as in the Raspberry PI 3 (ARM Cortex A53) the default distro is a pure 64bits Ubuntu Linux. While it would be theoretically possible to try to run the armv7 version with multilib, it would require quite a bit of extra disk space. Would it be possible to also include a 64bit release (arm64, armv8)? I sadly don't have a system available where I could try to compile it myself.

[mauritslamers]

Update: I managed to find a system where I could attempt to compile the arm64 version. I used the manual from https://github.com/LeonardLaszlo/nw.js-armv7-binaries/blob/master/docs/build-nwjs-v0.23.x.md. I managed to get rather far in the process but didn't manage to get a usable product. I found the following:

• in the manual there is a typo. Under step 1, the gclient line should read /nw23 at the end, not /nw22. Without this fix, the process fails at step 3, at the gclient sync line. (Fixed now)
• at step 3, all references to arm should be replaced by arm64. Moreover, nacl has to be disabled through enable_nacl=false under GN_ARGS, because the compilation fails otherwise.
• At the Build step, ninja -C out_gn_arm/nw dump fails with error strip: Unable to recognise the format of the input file

[mauritslamers]

Update: I managed to solve the strip problem. I found that the patches such as https://github.com/jtg-gg/node-webkit/commit/73f745bfb7587f38031d4e39a86718b64656bd5a need to be adjusted a bit in order to make things work under arm64. For every mention of arm as platform or current cpu, an additional mention of arm64 should be made. In src/nw/content/nw/tools/package_binaries.py, I added

                target['input'].append('nacl_irt_arm64.nexe')

under the existing arm line. Also in src/nw/content/nw/BUILD.gn the following should be added:

  else if (target_cpu == "arm64") {
    linux_strip_binary = "aarch64-linux-gnu-strip"
  }

and in src/nw/content/nw/nw.gypi the condition for arm stripping should also include the following:

            'target_arch="arm64"', {
              'variables': {
                'linux_strip_binary': 'aarch64-linux-gnu-strip',
                },
            },

This allows the process to almost finish. As mentioned above, the nacl support seems to be problematic in a similar way. The step gn gen out_gn_arm/nw --args="$GN_ARGS" fails because of missing definitions for arm64. Not including NaCl causes a problem for building the SDK version, as all the configuration scripts seem to automatically include NaCl when building the SDK.

[jtg-gg]

@mauritslamers thats great, nacl might not that important to many people so do you have a working build ?

[mauritslamers]

I think so, but I cannot create an SDK package to test because it requires nacl. I will try today to see whether I can adjust the packaging scripts to make an SDK without nacl.

[mauritslamers]

I managed to get a working arm64 package (added screenshot of a running nw.js on an arm64 platform). I adjusted the packaging script, commenting out all the lines where nacl was mentioned. In hindsight it is strange that the packaging script does not take the environment settings into account: even when explicitly disabling nacl during compilation it still tries to package it. What would be the best spot to upload the working arm64 version? [Edit: spelling]

[LeonardLaszlo]

I granted you collaborator rights. You can add your changes to the existing markdown files and modify the existing releases to include the 64 bit binaries.

[plokko]

Thank you @mauritslamers ! I was also struggling on getting an arm64 build with no results, i can't wait to try it out!

[mauritslamers]

I just updated the v0.23.7 release to include my arm64 build (thanks @LeonardLaszlo to grant me access!). As far as I can see, everything works as intended, but I didn't do test building. Also I assume that it should be renamed to -chrome-branding as I included the ffmpeg support in the build. Please keep me posted about your results.

[plokko]

It works on my Nanopi-k2 (Amlogic s905,on default image ubuntu-core xenial)! i had just to install few dependencies: sudo apt-get install -y libgconf-2-4 libxss1 thanks again!

[mauritslamers]

You're very welcome. In the end it was only possible because of the great building manual made by @LeonardLaszlo and the diffs created by @jtg-gg.

[TagLoomis]

Will we see a arm64 distribution for latest?

[LeonardLaszlo]

maybe one day if I get things done properly. First I'd like to automate the existing build process and use a CI system for the build. It is quite a hassle to do all these builds manually

[ubergeek77]

In the meantime, would @mauritslamers perhaps be able to provide some arm64 binaries for some of the later NW.js versions? I would be very grateful to get an arm64 binary for something like version 0.30.

[mauritslamers]

@ubergeek77 I tried to see whether I could build 0.30, but that didn't work out because the https://github.com/jtg-gg/chromium.src seems to be missing the 0.30 branch for the arm patches. Will have a go at 0.28

[mauritslamers]

Remarks about the arm64 build of 0.28:

I am not crosscompiling, but compiling on a Pine64 under arm64 linux. This brings some extra challenges.

• step 3 needs adjustment, we don't need target arch and target cpu definitions, as we are using the current platform.
• step 4 doesn't work, so you need to install the dependencies manually
• had to edit DEPS to get past the thirdparty/binutils/download.py, as it kept complaining about arm64 not being a supported platform (added and platform == "arm64" to the host == "linux" line)
• A few packages need separate compiling, which need clang.
• the depot_tools version of ninja doesn't run on arm64, this needs to be compiled manually as any installable versions are below 1.7.2 which is the required version in order to compile nw. Check out the repo with git clone https://github.com/ninja-build/ninja.git -b v1.8.2. Then cd ninja && ./configure.py --bootstrap. Copy the ninja executable from the root of the project to depot_tools/ninja
• the chromium buildtools don't bring an arm64 version of gn, so you need to compile gn. I got a problem that the linker flag --icf=all was not recognized, editing build/gen.py to comment out that block, solved this. After completing the compilation, copy the resulting gn binary to the src/buildtools/linux64 folder.
• the chromium buildtools don't bring an arm64 version of clang-format. Luckily this can be installed through apt-get, either symlink or copy the binary into src/buildtools/linux64

This gets us to the ninja compile step, which fails at the moment. More updates later.

[ubergeek77]

You're amazing! Thank you very much for working on this.

[mauritslamers]

Update:

More changes:

• the binaries in third_party/llvm-build/Release+Asserts/bin are not arm64 binaries, so install clang-6.0 through apt, and symlink in the executables
• the configuration of clang tries to include compilation plugins, which also don't run on arm64. I didn't find a way to compile those plugins or install them through apt, so I disabled the inclusion through editing build/config/clang/clang.gni and setting clang_use_chrome_plugins = false.

The ninja compile step now runs without immediately crashing. More updates later.

[mauritslamers]

Update:

More changes:

• the binaries in third_party/llvm-build/Release+Asserts/bin are not arm64 binaries, so install clang-6.0 through apt, and symlink in the executables
• the configuration of clang tries to include compilation plugins, which also don't run on arm64. I didn't find a way to compile those plugins or install them through apt, so I disabled the inclusion through editing build/config/clang/clang.gni and setting clang_use_chrome_plugins = false.

The ninja compile step now runs without immediately crashing.

After rather a while the compilation process fails because of a missing library, libXcursor.so.1 which it is searching for in the debian sysroot in build/linux/debian_stretch_arm64-sysroot/lib/aarch64-linux-gnu. The library is present on the system usually, in my case in /usr/lib/aarch64-linux-gnu. Symlinking libXcursor.so.1 in the debian sysroot, allows the compilation process to continue. More updates later.

[mauritslamers]

Update:

• step 1 and 2 of the compilation process did succeed. It takes a bit as I am using a Pine64 with 2GB of memory and an external HDD (no SSD) for swap and storage.
• step 3 fails because of the same compilation plugins mentioned above. Still hunting the cause.

[mauritslamers]

Update: Getting step 3 to compile is rather complex

• Compilation plugins: I tried the same approach by editing clang.gni, but that had no effect. I tried regenerating the ninja files, but the plugin kept being included. I ended up editing out the inclusion of that plugin from all the generated ninja files. This means removing -Xclang -load $NWJS/src/third_party/llvm-build/Release+Asserts/lib/libFindBadConstructs.so -Xclang -add-plugin -Xclang find-bad-constructs -Xclang check-ipc -g
• Wrong sysroot used: for some reason, on my system the ninja files in out_gn_arm/Release refer to a debian_jessie sysroot instead of the installed debian_stretch sysroot.
• libuv complains in compilation about a long long for a kernel constant which only works in C99. Changing --std=gnu89 to --std=gnu99 in $NWJS/src/out_gn_arm/Release/obj/third_party/node-nw/deps/uv/libuv.ninja solves this
• nghttp2 fails compiling because of implicit defined functions. This is caused by missing the HAVE_ARPA_INET_H=1 define (see https://github.com/nodejs/node/issues/14916). Adding this define to $NWJS/src/out_gn_arm/Release/obj/third_party/node-nw/deps/nghttp2/nghttp2.ninja fixes this
• Compilation finishes, but node-nw has a problem linking to nw, as for some reason nw.so doesn't exist in the build folder. More updates later

[mauritslamers]

Update: trying a few things in order to get libnw.so compiled. The only output currently is a directory full of libs, on which the 3rd compilation fails as it tries to link to libnw.so (through -lnw) which it cannot find as it does not exist.

[mauritslamers]

Update: I am still working on this, as compiling (and recompiling) on the Pine64 is very slow (only 2GB memory, and currently using usb-hdd for swap and storage. It also turns out that formatting an ext4 disk on ubuntu 18.04 and then using it under 16.04 causes issues, as ext4 under 18.04 uses checksums, which are not completely supported under 16.04. Once in a while it causes 16.04 to fail because of input/output errors, which can be solved by mounting it under 18.04, run fsck, then return it to the pine64 to continue. This might have to do with the pine64 running on an older kernel (3.10.104). I still have issues with getting libnw.so compiled. The out_gn_arm/nw/lib folder contains a libnw_base.so, but that obviously doesn't satisfy the node-nw compilation process.

Also working on an automated cross-compile build, using self hosted gitlab and gitlab-ci, as the entire compilation process takes multiple hours on modern hardware and services like Travis and AppVeyor only allow a maximum running time of one hour.

[aspect]

@mauritslamers what is your status on this? have you been successful with a cross-compile build? would trying to set up an arm64 VM help? https://wiki.ubuntu.com/ARM64/QEMU I can provide computing resources with SSH access (i.e. set up a VM on SSD and lots of RAM and provide SSH access) ... if this would help.

[janrinze]

Icestudio has migrated to nw.js 0.35. Any chance we will see nw.js 0.35 for armhf and aarch64? (i'd like both versions)

[LeonardLaszlo]

Unfortunately I don't have the time needed to tweak the builds.

[k1r0s]

sorry but TL;DR where I can download the bin in order to run it on arm64 systems?

[jrd261]

I just managed to cross-compile 0.51.3 for arm64 (no sdk, no nacl). Starts up on my Jetson ARM64 device but I haven't really tested much. I'll work on documenting the steps and providing a Dockerfile.

[LeonardLaszlo]

@jrd261 please update the building scripts and open a PR.

[jrd261]

It might take me a little to adapt my scripts as I used docker layer cache for the whole thing. Here's a synopsis on how I got it to work and maybe you guys will have some pointers (I don't know a whole lot about building with these tools). I'll work on getting this into a PR. I'll probably see if I can get an SDK version working first.

Installed sysroot with ./build/linux/sysroot_scripts/install-sysroot.py --arch=arm64

Patch to use correct sysroot:

--- third_party/node-nw/common.gypi
+++ third_party/node-nw/common.gypi
@@ -129,8 +129,8 @@
       ['OS=="linux" and target_arch=="x64" and <(building_nw)==1', {
         'sysroot': '<!(cd <(DEPTH) && pwd -P)/build/linux/debian_sid_amd64-sysroot',
       }],
-      ['OS=="linux" and target_arch=="arm" and <(building_nw)==1', {
-        'sysroot': '<!(cd <(DEPTH) && pwd -P)/build/linux/debian_sid_arm-sysroot',
+      ['OS=="linux" and target_arch=="arm64" and <(building_nw)==1', {
+        'sysroot': '<!(cd <(DEPTH) && pwd -P)/build/linux/debian_sid_arm64-sysroot',
       }],
       ['openssl_fips != ""', {
         'openssl_product': '<(STATIC_LIB_PREFIX)crypto<(STATIC_LIB_SUFFIX)',

Patch to have an arm64 target for cross-compilation

--- third_party/node-nw/common.gypi
+++ third_party/node-nw/common.gypi
@@ -580,9 +580,9 @@
             'cflags': [ '--sysroot=<(sysroot)', '-nostdinc++', '-isystem<(DEPTH)/buildtools/third_party/libc++/trunk/include', '-isystem<(DEPTH)/buildtools/third_party/libc++abi/trunk/include' ],
             'ldflags': [ '--sysroot=<(sysroot)','<!(<(DEPTH)/content/nw/tools/sysroot_ld_path.sh <(sysroot))', '-nostdlib++' ],
           }],
-          [ 'OS=="linux" and target_arch=="arm"', {
-            'cflags': [ '--target=arm-linux-gnueabihf' ],
-            'ldflags': [ '--target=arm-linux-gnueabihf' ],
+          [ 'OS=="linux" and target_arch=="arm64"', {
+            'cflags': [ '--target=aarch64-linux-gnu' ],
+            'ldflags': [ '--target=aarch64-linux-gnu' ],
           }],
           [ 'target_arch=="ppc" and OS!="aix"', {
             'cflags': [ '-m32' ],

Then I had to copy a bunch of [sysroot]/usr/include/aarch64-linux-gnu/* into [sysroot]/usr/include/. Not sure why the folder prefix is there.

GN Args

nwjs_sdk=false enable_nacl=false ffmpeg_branding=\"Chrome\" enable_widevine=false is_component_ffmpeg=true is_debug=false symbol_level=1 target_os=\"linux\" target_cpu=\"arm64\"

GYP_DEFINES=" nwjs_sdk=0 disable_nacl=1 enable_widevine=0 building_nw=1 buildtype=Official clang=1 OS=linux target_arch=arm64"

[pakimov2]

I just managed to cross-compile 0.51.3 for arm64 (no sdk, no nacl). Starts up on my Jetson ARM64 device but I haven't really tested much. I'll work on documenting the steps and providing a Dockerfile.

Great work! I also would like to run nwjs on my Jetson Nano. Can you provide binaries or instruction how to build? Thank you.