Can't no longer create macOS universal binary with @castlabs/electron-releases

[quanglam2807]

I received this error when building macOS universal binary with @castlabs/electron-releases v15.2.0

Error: Expected all non-binary files to have identical SHAs when creating a universal build but "Contents/Frameworks/Electron Framework.framework/Versions/A/Resources/MainMenu.nib/keyedobjects-101300.nib" did not

I checked the path, Contents/Frameworks/Electron Framework.framework/Versions/A/Resources/MainMenu.nib in the [email protected] and @castlabs/[email protected] is a file but is a directory in @castlabs/[email protected] with two files: keyedobjects.nib and keyedobjects-101300.nib.

Is there a way to fix this?

[khwaaj]

Interesting find. My current hunch is that this has something to do with the recent Xcode update that was installed on the macOS build machine, but I can't say for sure yet.

In any case, the files are essentially identical. What appears to have happened AFAICT is that the same data has been encoded differently into the NIBArchive structure stored in the keyedobjects-101300.nib files, perhaps due to different ordering of something like a dictionary object. Based on that assumption just picking one of them and use it for both should work as a temporary solution.

I'll try to investigate this further to see if we can do something at build time to make work as intended.

[khwaaj]

I can confirm that ibtool on the latest Xcode does this. Running xcrun ibtool --errors --warnings --notices --output-format human-readable-text --minimum-deployment-target 10.11.0 --compile MainMenu.nib MainMenu.xib, which is essentially what the Electron build system does in this case, will create a directory containing these two keyedobject files (whereas an older Xcode I have access to creates a single nib-file).

This appears to be non-derermenistic for the keyedobjects-101300.nib NIBArchive, so subsequent runs will create a slightly differing version of the same nib. This reinforces the assumption that these two files are actually the same, so just copying one over the other would be a viable workaround for now.

[quanglam2807]

Thank you for looking into this! Is it possible for you to update the files on your end?

[khwaaj]

Not for this release, no, but I'll see if we can figure something out for upcoming releases. I can't say for sure exactly when it will happen though, so in the mean time you would need to use the workaround I suggested.

I'd actually argue that ibtool should be deterministic in this case, the same way it is for the bplist version of the nib, but waiting for Apple to get that fixed may not be an option so I'll look into other alternatives too. I'll post any further updates in this ticket.

[sulf]

Sadly we have the same problem with a different file. We tried every binary from 15.x up to the latest release:

Expected all non-binary files to have identical SHAs when creating a universal build but "Contents/Frameworks/Electron Framework.framework/Versions/A/Resources/Electron Framework.sig" did not  
failedTask=build stackTrace=Error: Expected all non-binary files to have identical SHAs when creating a universal
 build but "Contents/Frameworks/Electron Framework.framework/Versions/A/Resources/Electron Framework.sig"
 did not

[khwaaj]

@sulf, check this comment out. In short, the .sig files are VMP signatures and they are safe to remove before creating your universal build (they would be invalidated anyway, and you would need to use EVS to re-sign).

Edit: Ideally some kind of mechanism for creating actions on a per file-type or path could be added to @electron/universal, which would allow easier handling of cases like this. I've been playing with some ideas around that but I've not had time to realize anything and submit a PR.

[sulf]

@khwaaj nice! Just kicked off a build with this to try. My understanding was that on macOS we need to use EVS to re-sign before we sign with the apple certificate? We're using electron-builder and I'm not sure where to plug the signing in. Currently I have the EVS signing in afterPack, which is called for every binary and for windows builds in afterSign.

[khwaaj]

Yes, you are right. And using afterPack on macOS and afterSign for Windows is the correct approach when using electron-builder.

[sulf]

@khwaaj Sadly afterPack isn't currently called for a universal binary on electron-builder. We ended up patching it.

The biggest issue we're having right now is that we cannot rebuild with electron-rebuild binaries anymore, since I don't see a valid way to provide headers pointing to your binaries. Want me to open up an issue for this here?

[khwaaj]

Well, that is a bummer. I actually never tested this with an universal build, so you would know better than I. It seems an afterPack call for universal builds would be something for electron-builder to support though, so maybe you should PR it?

Unfortunately we don't really have the resources to maintain compatibility with every type of setup out there, but for electron-rebuild you should be able to use the same headers as the corresponding stock Electron version, e.g. v16.0.4+wvcus -> v16.0.4. I've not actually done this but you might be able to use the --dist-url option to point to the right set of headers, e.g. https://www.electronjs.org/headers/v16.0.4/node-v16.0.4-headers.tar.gz, provided the documentation does what it says. Have you tried doing that?

[sulf]

Yeah, our team is preparing a PR for it. I'm sure they just forgot about it.

No need to change dist-url. Instead all we needed was to add a simple addition to package.json:

"build": {
  "electronVersion": "xxx"
},

Looks like that's picked up by electron-rebuild and works like a charm.

[mathieu2345]

This is still a problem for me. I am using Electron 18.0.4 and Xcode 13.3.1. Will there be a patch coming to Electron or Builder?

I have updated Electron-builder to the latest version.

[mathieu2345]

I can confirm that ibtool on the latest Xcode does this. Running xcrun ibtool --errors --warnings --notices --output-format human-readable-text --minimum-deployment-target 10.11.0 --compile MainMenu.nib MainMenu.xib, which is essentially what the Electron build system does in this case, will create a directory containing these two keyedobject files (whereas an older Xcode I have access to creates a single nib-file).

This appears to be non-derermenistic for the keyedobjects-101300.nib NIBArchive, so subsequent runs will create a slightly differing version of the same nib. This reinforces the assumption that these two files are actually the same, so just copying one over the other would be a viable workaround for now.

Might be a silly question but I am new to Mac - where would I locate the nib files? I have searched everywhere:(

[khwaaj]

The resulting NIB is generated in Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Resources/MainMenu.nib. And I'm pretty sure the latest version of electron-builder has fixed this problem (although the .sig file still needs to be handled on the side).

[mathieu2345]

I can confirm this was resolved in electron-builder 23.0.4. All good now.

[oliexe]

Any idea how to solve this on electron-builder 23.3.3, kinda blocking our release now. Thanks !

[khwaaj]

What exact issue are you referring to, the NIB problem or the signature file? If it is the NIB issue it was fixed by electron-builder upstream, but maybe there has been a regression.

[oliexe]

@khwaaj

Getting the following while building a universal app with the newest release of Electron for CS.

Expected all non-binary files to have identical SHAs when creating a universal build but "Contents/Frameworks/Electron Framework.framework/Versions/A/Resources/Electron Framework.sig" did not failedTask=build stackTrace=Error: Expected all non-binary files to have identical SHAs when creating a universal build but "Contents/Frameworks/Electron Framework.framework/Versions/A/Resources/Electron Framework.sig" did not

[khwaaj]

Yeah, that is because the included VMP signature files are created separately for x86 and arm64, so they will differ. The easiest fix is the one I mentioned previously in this discussion, i.e. to remove the .sig files from one or both of the builds before creating the universal package (e.g. in afterPack, see link in previous comment for an example). You'll need to regenerate the VMP signature anyway for your package since none of the separate signatures would be valid for the universal binary, so removing the files is safe.

[Mgrdich]

@khwaaj

a little tips for Electron Forge users who wants to have DRM and sign universal but make sure that their non universal build is not failing as well

do your macos normal arch x64 and arm64 inside the packageAfterExtract hook and do the fs.unlink here.

and the universal inside postPackage

do the

1. Widevide sign
2. osx-sign with the const { signAsync } = require('@electron/osx-sign');
3. osx-notarize with const { notarize } = require('@electron/notarize');

Because of the strcuture of the universal inside the electron-packager that are being used inside the electron-forge