SingleApplication blocks application on macOS with Apple M1

[csuft]

trafficstars

Hi, it's nice to find this open source project and I added this project in our source code. It works like charm both on Windows and macOS, except for macOS with Apple M1. We received some feedbacks from our client that the application won't be launched after installation. The log looks like the following:

It seems the application is blocked because of lock in shared memory. I don't know if anybody encountered this issue before. There is a similar issue in list #58 but I am not sure how to resolve it.

[itay-grudev]

I believe I read something, somewhere about a similar issue on apps running on M1s. Something about shared memory... Let me try to find it.

[itay-grudev]

From what I can see, there shouldn't be an issue with shared memory on M1 MacOS. But without having access to M1 hardware it would be hard for me to reproduce it. Any ideas? Do you have access to M1 hardware?

[itay-grudev]

One way would be to add an exception, but I'd rather investigate the real cause of the issue. For all I know it may be a Qt bug in the underlying implementation.

1. Can you try a more recent Qt version?
2. Can you try building your app on M1 hardware?

https://github.com/itay-grudev/SingleApplication/blob/d5b3852bd071f73bbbfc2cb802bdb98f9e7570ef/singleapplication.cpp#L44-L49

[csuft]

Yeah, I compiled my project on M1 mac mini(Version 11.2), and the Qt version is 5.15.2. I don't have too much to compile other versions and currently we don't have any plans to migrate to Qt 6.

[itay-grudev]

I need assistance for this as I currently don't have access to an M1 machine and cannot look into it in dept. But my guess is there is some Qt issue on that platform.

[HealsCodes]

Hey, I'm happy to help if I can - I'm using SingleApplication for one of my projects and I'm working solely on an M1 MacBook Air.

For a starter I'm not observing the problem with using SingleApplication with allowSecondary=false under Qt6 arm64. So if it was a Qt bug then it has been fixed.

However if I try to compile against an arm64 version of Qt5 I also can't reproduce the issue. So it seems like the problem might be related to using the official x86_64 Qt5 build under Rosetta2 on an M1.

(note: @csuft that Qt5 isn't officially supported natively on M1 so your applications will have to rely on Rosetta2 x86_64 translation which will eventually be removed in the future).

[itay-grudev]

@csuft Is the problem still present in a build with the latest Qt5 version, to confirm that the issue is gone as @Shirk suggests?

[itay-grudev]

I am closing the issue for now. If there is new feedback on this - please feel free to share it and we could reopen it.

[danryu]

I'm seeing this issue now, when trying to run a Universal (Intel+AppleSilicon) app built using Qt 6.3.2, running on an intel MacBook.

When I run the app from the terminal I simply see:

SingleApplication: Unable to create block.
SingleApplication:  1 "QSharedMemoryPrivate::initKey: unable to set key on lock"

Is the M1-2/arm support not likely to work? I was hoping to deploy singleapplication for our mac users ... (app here: https://github.com/koord-live/koord-app )

[itay-grudev]

I am going to need help with this as I don't have access to an M1 machine.

Would it be possible for you to investigate and submit a PR?

On Thu, 27 Oct 2022, 16:13 danryu, @.***> wrote:

I'm seeing this issue now, when trying to run a Universal (Intel+AppleSilicon) app built using Qt 6.3.2 on an intel MacBook.

When I run the app from the terminal I simply see:

SingleApplication: Unable to create block. SingleApplication: 1 "QSharedMemoryPrivate::initKey: unable to set key on lock"

Is the M1-2/arm support not likely to work? I was hoping to deploy singleapplication for our mac users ... (app here: https://github.com/koord-live/koord-app )

— Reply to this email directly, view it on GitHub https://github.com/itay-grudev/SingleApplication/issues/136#issuecomment-1293508626, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQGP534VAS5CENSLQ6KB6TWFJ5YBANCNFSM5BQCD4XQ . You are receiving this because you modified the open/close state.Message ID: @.***>

[danryu]

I also don't have access to an M1 machine. I can help debug this use case (running a Universal build on intel mac) but I don't have project bandwidth or expertise to start modifying SingleApplication.

[danryu]

Is it necessary to actually have a physical M1 mac, if I can reproduce the error by running the Universal build on intel?

Otherwise is this a Won't Fix scenario? Which would be a bummer as M1 macs are only gonna get more popular, and I really wanted SingleApplication working everywhere...

[HealsCodes]

I - apparently - have access to an M1, what would be required to reproduce this?

(and to answer the question: M1 bugs are CPU-specific, the M1 macOS environment differs a lot from an intel* build and the intel slice of a universal binary is completely independent in it's behaviour)

[danryu]

I - apparently - have access to an M1, what would be required to reproduce this?

The universal build that fails for me on Intel will probably also fail for you on M1: https://github.com/koord-live/koord-app/releases/download/r4_0_16pre6/Koord_4.0.16_.dmg

If you install and run from Terminal, you will likely the same error I posted above.

(and to answer the question: M1 bugs are CPU-specific, the M1 macOS environment differs a lot from an intel* build and the intel slice of a universal binary is completely independent in it's behaviour)

I'm not sure this is an "M1 bug" or what one of those is exactly. I can say that the Universal build dmg installer is 39Mb in size, and the individual intel/M1 ones are 37Mb - only 2Mb smaller. Also running the Universal build running on Intel seems to fail with the same error as OP on M1, so they can't be that independent. (Btw this is a blocker for submitting to App Store as Universal build is required to fully support both CPU archs for users.)

[danryu]

@Shirk Maybe the simplest reproduction would be if you ran the simplest example in QtCreator: https://github.com/itay-grudev/SingleApplication/tree/master/examples/basic

If that works/fails that would provide a good baseline for debugging :)

[HealsCodes]

I'm not sure this is an "M1 bug" or what one of those is exactly. I can say that the Universal build dmg installer is 39Mb in size, and the individual intel/M1 ones are 37Mb - only 2Mb smaller. Also running the Universal build running on Intel seems to fail with the same error as OP on M1, so they can't be that independent. (Btw this is a blocker for submitting to App Store as Universal build is required to fully support both CPU archs for users.)

You're looking at a DMG.. which is a compressed archive file and has zero indication of it's content sizes. The Koord.app is over 200MB as universal binary and most of that is Qt.

If you remove the unused architecture part you can se that almost everything is different:

 /tmp % du -sh Koord.app
283M	Koord.app
 /tmp % du -sh Koord_thin_arm64.app
140M	Koord_thin_arm64.app
 /tmp % du -sh Koord_thin_intel.app
149M	Koord_thin_intel.app

The Problem is also not with QSingleApplication but rather with your codesigning entitlements which prevent shared memory usage, which in turn causes QSharedMemoryPrivate::initKey to fail.

You can test this by "dummy signing" your application without a sandbox: codesign -s - -f --deep Koord.app/

Which produces a working application (because no sandboxing is enforced) I'm still checking which entitlements key is required but it's fair to say that this is out-of-scope for QSingleApplication.

[HealsCodes]

Update - more research - still not QSingleApplication's fault; Qt has a number of "specialities" for Apple & the Mac AppStore:

• you need to build Qt6 yourself and make sure to use the "-appstore-compliant" flag as per QTBUG-106910
• Shared memory has to be created in the form "DEVTEAMID.app-group/shared"
  • QSingleApplication could do that, but it would have to know about the fact in the first place
• your app might require an associated app group on Apples profiles & identifiers page (see Qt6 - QSharedMemory on Apple Platforms)

[itay-grudev]

Ahaa! This then is totally not what I expected. Maybe I should add a link to that Qt article with specifics for MacOS.

On Thu, 27 Oct 2022, 19:28 Renée Köcher, @.***> wrote:

Update - more research - still not QSingleApplication's fault; Qt has a number of "specialities" for Apple & the Mac AppStore:

• you need to build Qt6 yourself and make sure to use the "-appstore-compliant" flag as per QTBUG-106910 https://bugreports.qt.io/browse/QTBUG-106910
• Shared memory has to be created in the form "DEVTEAMID.app-group/shared"
  • QSingleApplication could do that, but it would have to know about the fact in the first place
• your app might require an associated app group on Apples profiles & identifiers page (see Qt6 - QSharedMemory on Apple Platforms https://doc.qt.io/qt-6/qsharedmemory.html#details)

[image: Screenshot 2022-10-27 at 18 23 @.*** https://user-images.githubusercontent.com/304352/198345918-8a1f8612-2cc6-4cc4-972b-1db1f1e9b0d3.png

— Reply to this email directly, view it on GitHub https://github.com/itay-grudev/SingleApplication/issues/136#issuecomment-1293782683, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQGP52IPI5IBJJW7DDVPTDWFKUUBANCNFSM5BQCD4XQ . You are receiving this because you modified the open/close state.Message ID: @.***>

[itay-grudev]

We can also change the name of the shared memory key automatically. I'll read more into this. This may have been the missing link, but I have no way to test any changes reliably.

On Thu, 27 Oct 2022, 19:31 Itay Grudev, @.***> wrote:

Ahaa! This then is totally not what I expected. Maybe I should add a link to that Qt article with specifics for MacOS.

On Thu, 27 Oct 2022, 19:28 Renée Köcher, @.***> wrote:

Update - more research - still not QSingleApplication's fault; Qt has a number of "specialities" for Apple & the Mac AppStore:

• you need to build Qt6 yourself and make sure to use the "-appstore-compliant" flag as per QTBUG-106910 https://bugreports.qt.io/browse/QTBUG-106910
• Shared memory has to be created in the form "DEVTEAMID.app-group/shared"
  • QSingleApplication could do that, but it would have to know about the fact in the first place
• your app might require an associated app group on Apples profiles & identifiers page (see Qt6 - QSharedMemory on Apple Platforms https://doc.qt.io/qt-6/qsharedmemory.html#details)

[image: Screenshot 2022-10-27 at 18 23 @.*** https://user-images.githubusercontent.com/304352/198345918-8a1f8612-2cc6-4cc4-972b-1db1f1e9b0d3.png

— Reply to this email directly, view it on GitHub https://github.com/itay-grudev/SingleApplication/issues/136#issuecomment-1293782683, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQGP52IPI5IBJJW7DDVPTDWFKUUBANCNFSM5BQCD4XQ . You are receiving this because you modified the open/close state.Message ID: @.***>

[danryu]

Update - more research - still not QSingleApplication's fault; Qt has a number of "specialities" for Apple & the Mac AppStore:

I don't need to do any of that normally. The Koord app has been building and running fine for ages, and also accepted without problem into App store. None of the links you specified are necessary. I certainly don't need to build Qt6 myself to create a working Mac installer, either for direct download (and notarization) or validation and upload to app store.

[HealsCodes]

Update - more research - still not QSingleApplication's fault; Qt has a number of "specialities" for Apple & the Mac AppStore:

I don't need to do any of that normally. The Koord app has been building and running fine for ages, and also accepted without problem into App store. None of the links you specified are necessary. I certainly don't need to build Qt6 myself to create a working Mac installer, either for direct download (and notarization) or validation and upload to app store.

Well.. your binary is completely valid and thus can be code-signed and uploaded to the mac-appstore, but the shared memory QSingleApplication uses won't work because it does not follow the requirements for sandboxed applications. This is a runtime-requirement, so it doesn't trip anything during Apple's validations as they don't run the binaries when notarising but only scan symbols and signatures.

[danryu]

@itay-grudev Seems like @Shirk has isolated the issue, has an M1 Mac and is willing to run test builds :) Do we have the elements to potentially implement a fix?

[itay-grudev]

@danryu

The only thing I can do is to address the following points:

• The key must be in the form <application group identifier>/<custom identifier>, as documented here and here.
• The key length is limited to 30 characters.

Currently the shared memory uses the following as a key:

https://github.com/itay-grudev/SingleApplication/blob/f1e15081dc57a9c03f7f4f165677f18802e1437a/singleapplication_p.cpp#L131-L185

This code basically computes a hash based on the input data for single application (like application name, organisation, username, etc.) and computes a hash to be used as a shared memory identifier. I suppose we will have to use a different one that follows the scheme specified above.

Just one question - is the <application group identifier> something Apple assigns developers or could it be arbitrary?

[danryu]

Just one question - is the <application group identifier> something Apple assigns developers or could it be arbitrary?

I'm not 100% on this - but I think the application group identifier corresponds to the PRODUCT_BUNDLE_IDENTIFIER which in the case of Koord is live.koord.Koord-RT. https://doc.qt.io/qt-6/qmake-variable-reference.html#qmake-info-plist

(I think the Team-id is an alphanumeric token which Apple applies automatically in App Store Connect.)

[HealsCodes]

Just one question - is the <application group identifier> something Apple assigns developers or could it be arbitrary?

I'm not 100% on this - but I think the application group identifier corresponds to the PRODUCT_BUNDLE_IDENTIFIER which in the case of Koord is live.koord.Koord-RT. https://doc.qt.io/qt-6/qmake-variable-reference.html#qmake-info-plist

It doesn't - that's your application identifier, the application group identifier is something you create separately here:

App Groups always start with a "group." prefix

You have to enable "App Groups" when creating the application identifier for you app and then select the group in order for the app to be allowed access: (also here)

(I think the Team-id is an alphanumeric token which Apple applies automatically in App Store Connect.)

Correct, you get assigned one Team-ID per account. It is randomly generated by Apple but does not change between applications.

There's no way - I'm aware of - for QSingleApplication to determine these values other than the developer providing them at configuration time or via preprocessor defines.

I hope this info helps.

[danryu]

Many thanks @Shirk - this is indeed very helpful. I'll go ahead and create the Application Group as advised.

There's no way - I'm aware of - for QSingleApplication to determine these values other than the developer providing them at configuration time or via preprocessor defines.

That I can do fairly easily I suppose. Not sure @itay-grudev how best this could/would be integrated with SingleApplication?

(PS Apple certainly don't make it easy in this case do they? But I guess this support will only be more necessary in the future as Intel builds get phased out...)

[HealsCodes]

Many thanks @Shirk - this is indeed very helpful. I'll go ahead and create the Application Group as advised. You're welcome

(PS Apple certainly don't make it easy in this case do they? But I guess this support will only be more necessary in the future as Intel builds get phased out...)

Yeah, they have most of it fully automated if you use XCode which even manages application IDs, group identifiers and signing entitlements for you but if you don't want to or can't use XCode it's a lot of documentation and forum knowledge to absorb and a lot of trial and error involved.

(I only learned most of those things in the first place because it equally applies to iOS and has for years)

[danryu]

Well, I've tried all the steps advised, but I still can't get it to work :( The app crashes still startup with this message:

SingleApplication: Unable to create block.
SingleApplication:  1 "QSharedMemoryPrivate::initKey: unable to set key on lock"

Steps followed:

• built Qt 6.3.2 for Mac with Posix compliance (script here)
• created Application Group Id group.koord in Apple Developer Center
• set blockServerName = "TXZ4FR95HG.koord/shared"; in singleapplication_p.cpp to override the logic in genBlockServerName() and to follow the pattern of QSharedMemory shm("DEVTEAMID.app-group/shared"); given at https://doc.qt.io/qt-6/qsharedmemory.html#details.

It's not yet clear to me from all the docs linked whether I have done everything right - in particular I'm not sure whether to use the "group." prefix on the Application Group id. It gets added automatically in Developer Center, but the Apple docs here suggest not to use it. I may be wrong though.

Any help appreciated!

[HealsCodes]

Well, I've tried all the steps advised, but I still can't get it to work :( The app crashes still startup with this message:

SingleApplication: Unable to create block.
SingleApplication:  1 "QSharedMemoryPrivate::initKey: unable to set key on lock"

Steps followed:

• built Qt 6.3.2 for Mac with Posix compliance (script here)
• created Application Group Id group.koord in Apple Developer Center
• set blockServerName = "TXZ4FR95HG.koord/shared"; in singleapplication_p.cpp to override the logic in genBlockServerName() and to follow the pattern of QSharedMemory shm("DEVTEAMID.app-group/shared"); given at https://doc.qt.io/qt-6/qsharedmemory.html#details.

It's not yet clear to me from all the docs linked whether I have done everything right - in particular I'm not sure whether to use the "group." prefix on the Application Group id. It gets added automatically in Developer Center, but the Apple docs here suggest not to use it. I may be wrong though.

You are correct, the docs state that the "group" prefix is only required on iOS, your TXZ4FR95HG.koord/shared looks fine to me. As does the group selection. Just to make sure, did you recreate and re-download the signing profile for the App-ID afterwards? (My terms might be a bit off, for iOS your signing certificates are paired with a provisioning profile which in turn is bound to the application ID and links it to details like allowed application groups. It feels like you do everything correctly bu t macOS just doesn't know your app belongs into that group because it can't link it to a provisioning see here)

[danryu]

Thanks @Shirk. I did re-create the iOS provisioning profile, but not yet for macOS. I will do this - but also need to remind myself how to apply the provisioning profile during the build with Qt. Not sure if I can use any options in macdeployqt or productbuild to do this...