Provide pre-built downloads for ARM64 architecture for Electron

[rathboma]

I run Beekeeper Studio and I want to move to better-sqlite3 over sqlite3 due to a number of issues (you were right after all, it is better :-).

As part of the electron-builder build process it downloads pre-built binaries. This works great for Linux x64, but it'd be great to also provide an arm64 build.

Would that be possible? Not sure if you use Github Actions for build, but happy to give you access to my arm64 AWS box if you want to share it.

[CyberSecDemon]

I second this.

[nsantos16]

Same problem here, happy to help to incorporate this on the github action

[matthewcoxx]

We've actually had to switch our development back to node-sqlite3 because this has persisted along with node miss match errors when building in electron.

[nsantos16]

@rathboma Can we use your AWS box build? cc: @JoshuaWise

[baryon]

How build it from source when no prebuild binanry on M1 mac

[baryon]

To ensure your native dependencies are always matched electron version, simply add script `"postinstall": "electron-builder install-app-deps" to your `package.json`
  • writing effective config  file=dist/builder-effective-config.yaml
  • rebuilding native dependencies  [email protected], [email protected] platform=darwin arch=arm64
  • install prebuilt binary  name=keytar version=7.7.0 platform=darwin arch=arm64 napi= 
  • install prebuilt binary  name=better-sqlite3 version=7.4.1 platform=darwin arch=arm64 napi=
  • build native dependency from sources  name=better-sqlite3
                                          version=7.4.1
                                          platform=darwin
                                          arch=arm64
                                          napi=
                                          reason=prebuild-install failed with error (run with env DEBUG=electron-builder to get more information)
                                          error=prebuild-install info begin Prebuild-install version 6.1.2
    prebuild-install WARN install prebuilt binaries enforced with --force!
    prebuild-install WARN install prebuilt binaries may be out of date!
    prebuild-install info looking for local prebuild @ prebuilds/better-sqlite3-v7.4.1-electron-v89-darwin-arm64.tar.gz
    prebuild-install info looking for cached prebuild @ /Users/xxx/.npm/_prebuilds/993ab9-better-sqlite3-v7.4.1-electron-v89-darwin-arm64.tar.gz
    prebuild-install http request GET https://github.com/JoshuaWise/better-sqlite3/releases/download/v7.4.1/better-sqlite3-v7.4.1-electron-v89-darwin-arm64.tar.gz
    prebuild-install http 404 https://github.com/JoshuaWise/better-sqlite3/releases/download/v7.4.1/better-sqlite3-v7.4.1-electron-v89-darwin-arm64.tar.gz
    prebuild-install WARN install No prebuilt binaries found (target=13.1.2 runtime=electron arch=arm64 libc= platform=darwin)
    
  • rebuilding native dependency  name=better-sqlite3 version=7.4.1

it stoped at rebuilding status over one hour. whats happen? how fix it?

[baryon]

I found why rebuild stopped. ./node_modules/.bin/electron-builder build --mac --arm64 works npx electron-builder build --mac --arm64 stopped

[rathboma]

@JoshuaWise email me and I'll give you a sudo enabled login to my AWS arm box if you want it. First name dot lastname Gmail. :-)

[jackple]

also win32-ia32 with Electron

[cutmoney]

Any update on this? I could really use a pre-build binary for arm64 as well. Or, alternatively, is there any way to avoid rebuilding the module every time? I'd sure like reducing the 5-10 minute builds every time on my Pi 4.

[nsantos16]

Same question here, any update or help you need about this? 🤔

[dennisameling]

Just chiming in here - it'd probably be most future-proof if this project switches to N-API first. Currently, there's already 26 prebuilds for every release, and every new Electron or Node version adds to that. When adding additional platforms/architectures to the mix, that number will once again increase exponentially.

We did the same in node-keytar and it reduced the amount of prebuilds from 92 (!) to 12.

A large part (but not all) of the N-API migration can be handled by its conversion tool. I've blocked some time towards the end of this month to look into that 👍🏼

[cutmoney]

On a related side note, does anyone know if there's an option in electron-forge to prevent natively rebuilding the module every time? I realize that electron-builder has that option, but that's erroring out for me on my Pi build for some reason.

[alagrede]

I found why rebuild stopped. ./node_modules/.bin/electron-builder build --mac --arm64 works npx electron-builder build --mac --arm64 stopped

Hello @baryon, Did you succeed to compile better-sql for arm64 apple M1 and use it? On my side, it seems to crash at runtime with a no suitable image found while importing Database Thanks for your help

[baryon]

Yes, no problem here

1,check your node versison, if you use M1, update node to the latest version 16

% node -v
v16.10.0

2,write package.json like

{
.... ....
  "scripts": {
    "build:mac:arm64": "DEBUG=electron-builder ./node_modules/.bin/electron-builder build --mac --arm64",
    "postinstall": "./node_modules/.bin/electron-builder install-app-deps"
  },
.... ....

  "devDependencies": {
    "electron": "^13.5.1",
    "electron-builder": "^22.14.5",
    "electron-notarize": "^1.1.1",
    "electron-osx-sign": "^0.5.0"
.... ....
  },
  "dependencies": {
    "better-sqlite3": "^7.4.3"
.... ....
  }
}

3, npm install or yarn 4, npm run build:mac:arm64

[adambailey-]

@JoshuaWise Now that Node 16 is LTS, would it be possible for a new version to be released with ARM64 Electron prebuilds? I would love to be able to fully swap to this from node-sqlite3, but the prebuilds are holding me up

[BugInACode]

I too am being held up, every time I try to build an app I'm try to port across from x64 to arm64 I keep having to rebuild this module but it seems to fail... a prebuilt binary would definitely help me too

[cutmoney]

Same here. An arm64 binary would be extremely helpful!

[adambailey-]

@JoshuaWise Is there any sort of update you could possibly provide for this? I'm sure you're busy, and with the holidays coming up, it might get even harder to get this done 😅 . If you could provide any sort of rough guess on when you expect to be able to take this on, it would be greatly appreciated, even if the answer is that it will be months

[BugInACode]

Alternatively to doing this for us, @JoshuaWise where could we find the build instructions? We might be able to give it a shot ourselves to ease some of the work off you.

[cbartondock]

I would like to chime in that binaries for linux ia32 (electron and node) would be very helpful.

[gpetrov]

A great example of optimal binary distribution is the Node Sharp project, using N-API v5 and github actions, resulting in just 8 binaries for all OS's that can be used for Electron or just bare NodeJS.

https://github.com/lovell/sharp/tree/master/.github/workflows

[cbartondock]

I ended up just forking this project and modifying the build actions to make the binaries I need. My actual project now downloads its binaries from there. To do the linux ia32 binaries it was necessary to first apt-get install g++-multilib and gcc-multilib, but otherwise simple.

[adambailey-]

Nice! Would it be pretty simple to put together a PR with those changes onto this repo?

[cbartondock]

Yes although I made another somewhat hacky change in able to get the library to work when called from electron's renderer process like we are always told never to do 👀. I could make a clean fork that only changes the build actions though. The additional action is:

   - if: matrix.os == 'ubuntu-latest'
        run: |
          sudo apt-get update
          sudo apt-get install gcc-multilib g++-multilib
          npx --no-install prebuild -r electron -t 10.0.0 -t 11.0.0 -t 12.0.0 -t 13.0.0 -t 14.0.0 -t 15.0.0 -t 16.0.0 --include-regex 'better_sqlite3.node$' --arch ia32 -u ${{ secrets.GITHUB_TOKEN }}

[BugInACode]

@cbartondock I tried adding your repo to my package.json file but now it fails to run yarn grunt. Did you mind expanding on what you did that allowed you to use your fork? Thanks in advance

[cbartondock]

Other than changes to the build actions all I did was this: https://github.com/cbartondock/better-sqlite3/commit/5b1a7901a01c296aa0d21a3e9a87d9168794f66f

To make it compatible with my typescript stack. That said I made a lot of changes to the build.yml file and removed many builds I was not using to save time, so potentially that could be causing your issue as well.

The relevant code that needs to be added to get it to build linux ia32 is above.

[BugInACode]

This is actually a dependency of an app I'm trying to build, so in the package.json file I've changed the url to your fork, then delete the yarn.lock file and run yarn install but fails. Should I build this dependency separately then try building my app? Thanks

[JoshuaWise]

Perhaps this is a low-hanging fruit, now that https://github.com/JoshuaWise/better-sqlite3/pull/714 is merged. PRs welcome.

[tkurki]

Can't easily test this as a PR, but a cropped up version of

  prebuild-armv7-alpine:
    name: Prebuild on armhf alpine
    runs-on: ubuntu-latest
    needs: publish
    steps:
      - uses: docker/setup-qemu-action@v1
      - run: |
          docker run --rm --entrypoint /bin/sh --platform linux/arm/v7 node:16-alpine -c "apk add build-base git python3 --update-cache && \
          git clone ${{ github.event.repository.clone_url }} && \
          cd ${{ github.event.repository.name }} && \
          npm install --ignore-scripts && \
          npx --no-install prebuild -r node -t 10.20.0 -t 12.0.0 -t 14.0.0 -t 16.0.0 --include-regex 'better_sqlite3.node$' -u ${{ secrets.GITHUB_TOKEN }}

seems to build ok. How would one go about testing the result?

Having prebuilts is really a must for adoption on RPi, people tend to report stuck installs when it just takes forever.

I can submit a PR blindly if that helps.