Building issue

[shirshak55]

I am unable to build because the server isn't available for MacBook silicon.

Here is the error.

npm run build

> [email protected] build
> npm run build:src && npm run build:electron


> [email protected] build:src
> tsc


> [email protected] build:electron
> ts-node ./setup-server.ts && electron-builder build

Downloading httptoolkit-server v1.9.1 for darwin-arm64
Error: No server available matching /httptoolkit-server-v1.9.1-darwin-arm64.tar.gz/
    at /Users/quantum/Desktop/projects/mike/httptoolkit-desktop/setup-server.ts:78:15
    at Generator.next (<anonymous>)
    at fulfilled (/Users/quantum/Desktop/projects/mike/httptoolkit-desktop/node_modules/tslib/tslib.js:112:62)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

[pimterry]

Hmm, yes, that makes sense.

Right now there's no build for Mac Silicon, mostly because I don't have a test device, but also because it doesn't seem essential. As far as I can tell, the existing version works fine, and any minor performance hit shouldn't matter much, since HTTP Toolkit is not especially performance-sensitive software and these are very fast laptops.

For now, there's two options: you can use a quick workaround, or if you're feeling keen we can try and fix this properly.

There's at least two workaround options I think:

• If you run your whole terminal via emulation (I think arch -x86_64 /bin/bash should do this?) my understanding is that it will automatically emulate every subprocess, so it'll automatically work immediately.
• Alternatively, you can modify the setup-server.ts script to just pull the darwin-x64 version when running on arm64, and then that should automatically emulate the server when it's launched, so that'll work.

On the other hand, if you're interested in fixing this for real, that would be a huge help, since you clearly have a test machine to do so and I don't. I think the steps are something like:

• Manually run the server on arm64 and check it works native, without emulation. There's some native dependencies in here, but I think the few binaries that are relevant on Mac have builds available for arm64 already.
• Work out how to build an arm64 package for the server for Mac. It might just be a matter of setting npm_config_arch and npm_config_target_arch like we do to cross-compile for Windows & Mac from Linux (here) and adding another build option to use that here
• Manually test the built ARM package, make sure the result really does work happily, running natively

If we did that, the desktop here would run locally fine for development I think. To then publish a 100% ARM desktop build, we'd also need to switch to a universal electron app, and work out how to build that in CI. That's harder I think, because GitHub don't have ARM Mac runners yet, and we don't currently do any cross-building for desktop - we run a separate job for each platform (all x64) and each one builds its own native desktop app.

Just switching the server would be a good step forwards though, would solve your problem, and would probably have some nice easy performance improvements!

[shirshak55]

@pimterry Sorry for the delay.

I think mac silicion GitHub runner is now available. Should we look at it?

[pimterry]

@shirshak55 great to hear from you!

Yes, I'm definitely interested in using the new GitHub runner, but notably it's not free for public repos unlike all other runners - it's $0.16 per minute. That means right now, it'd be approximately $3 every build for both this repo and the server, if the whole build ran via M1 (maybe it would also run quicker, but for this repo at least half the time is just waiting for Apple notarization). I did a quick test in the server and it won't run unless I allow billing.

Not the end of the world by any means, but it'd be easy to rack up big charges without noticing while we're testing (especially if we break something, and builds accidentally run until the 4 hour timeout, which would be a $40 charge every time) so I'd like to be careful there.

We could certainly start getting builds working fully outside GitHub first though, and then set up the GH part to automate that later once it's stable. Later on we can try to limit how much of that has to run on an actual M1 too, but let's not worry about that right now.

---

So next steps:

What is your testing setup? Do you have Linux running on your M1, or Mac, or both?

I forget the exact state of the server, but I think the server build currently works on Mac, and it's just that we're not auto-building & publishing the ARM64 version? That would be the next step here if so (and that doesn't require an M1 for the build - the server build works by building everything on Linux and cross-installing prebuilt native dependencies).

Once that works, we can make any changes in the desktop here required to do an M1 builds, which should just pull the latest ARM64 server and use an ARM64 Electron runtime, and then we're all good.

Let's focus on Mac M1 first, but I'd also love to do Linux ARM64 latest since that's been requested a few times. The main problem was node-datachannel, but in fact the next version (currently still pre-release) has NAPI builds that include Linux arm & arm64 as well, so we should be able to do that too soon!

[shirshak55]

I have mac m2. I don't have any linux, however I can easily rent digital ocean server.

Thanks for the next steps. I will start workin' on those.

[pimterry]

Great! Thanks for looking into this :smile:. Let me know if there's anything else I can do to help, or if you have any questions at all.