[scripts|cmake] Fix up install name and rpath on osx dynamic

[m-kuhn]

Picks up the work in https://github.com/microsoft/vcpkg/pull/32200

Original description by @liviaerxin

It's supposed to uniform dynamic libraries and executable install name and rpath on osx of arm64-osx-dynamic triplet. Due to many reasons, some individual ports may set the wrong rpath and install name on osx, or not set them. Here's hoping to manage them in a standard way in vcpkg.

It is supposed to solve general problems like #31719, continuing on #31720, also referring #23035

Here's the rule which may be quite different than fixing rpath mentioned at #23035 as this following:

After building and installing a package into /opt/vcpkg/packages/xxxx_arm64-osx-dynamic, it'll do the fix on Mach-O files in lib/, debug/lib/, tools/xxx/, following:

1. for shared libraries in lib/ and debug/lib/:

   • fix install name on itself to be with @rpath prefix

   • fix install name in its dependent shared libraries to be with @rpath prefix

     • include libraries only from the same port, assuming other libraries from other managed ports by vcpkg are already fixed!
     • exclude system libraries in /usr/lib/ and /System/Library!

   • fix rpath on itself to be @loader_path currently (Or using absolute path CURRENT_INSTALLED_DIR, like /opt/vcpkg/installed/arm64-osx-dynamic/lib or /opt/vcpkg/installed/arm64-osx-dynamic/debug/lib?)

2. for exe in tools/{package}/bin/

   • fix install name in its dependent shared libraries to be with @rpath prefix

     • include libraries only from the same port in lib/, assuming other libraries from other managed ports by vcpkg are already fixed!
     • exclude system libraries in /usr/lib/ and /System/Library!

   • fix rpath to be @loader_path/../../../ to be relative with lib/(Or using absolute path CURRENT_INSTALLED_DIR, like /opt/vcpkg/installed/arm64-osx-dynamic/lib?)

Have done a test on [qtbase] and others

Before:

❯ ~/Documents/dlopen_test.out /opt/vcpkg/installed/arm64-osx-dynamic/tools/Qt6/bin/rcc
dlopen(/opt/vcpkg/installed/arm64-osx-dynamic/tools/Qt6/bin/rcc, 0x0001): Library not loaded: @rpath/libQt6Core.6.dylib
  Referenced from: <28854ECA-06DA-351C-81B3-C922A23B6832> /opt/vcpkg/installed/arm64-osx-dynamic/tools/Qt6/bin/rcc
  Reason: tried: '/opt/vcpkg/installed/arm64-osx-dynamic/tools/Qt6/bin/../lib/libQt6Core.6.dylib' (no such file), '/opt/vcpkg/installed/arm64-osx-dynamic/tools/Qt6/bin/../lib/libQt6Core.6.dylib' (no such file), '/usr/local/lib/libQt6Core.6.dylib' (no such file), '/usr/lib/libQt6Core.6.dylib' (no such file, not in dyld cache)
dlopen sample%                                                                                                                         

❯ otool -L /opt/vcpkg/installed/arm64-osx-dynamic/debug/lib/libQt6Core_debug.6.dylib
/opt/vcpkg/installed/arm64-osx-dynamic/debug/lib/libQt6Core_debug.6.dylib:
	@rpath/libQt6Core_debug.6.dylib (compatibility version 6.0.0, current version 6.5.1)
	/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1971.0.0)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1971.0.0)
	/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 2299.50.120)
	/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 61.0.0)
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1228.0.0)
	/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60420.101.2)
	/opt/vcpkg/installed/arm64-osx-dynamic/debug/lib/libb2.1.dylib (compatibility version 2.0.0, current version 2.4.0)
	@rpath/libicui18n.73.dylib (compatibility version 73.0.0, current version 73.1.0)
	@rpath/libicuuc.73.dylib (compatibility version 73.0.0, current version 73.1.0)
	@rpath/libicudata.73.dylib (compatibility version 73.0.0, current version 73.1.0)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
	@rpath/libdouble-conversion.3.dylib (compatibility version 3.0.0, current version 3.2.0)
	@rpath/libpcre2-16.0.dylib (compatibility version 12.0.0, current version 12.2.0)
	/opt/vcpkg/installed/arm64-osx-dynamic/debug/lib/libssl.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	/opt/vcpkg/installed/arm64-osx-dynamic/debug/lib/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	@rpath/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.5)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1500.65.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

After fix:

❯ ~/Documents/dlopen_test.out /opt/vcpkg/installed/arm64-osx-dynamic/tools/Qt6/bin/rcc
dlopen sample%                                                                                                                                                                                                                                       

❯ otool -L /opt/vcpkg/installed/arm64-osx-dynamic/debug/lib/libQt6Core_debug.6.dylib
/opt/vcpkg/installed/arm64-osx-dynamic/debug/lib/libQt6Core_debug.6.dylib:
	@rpath/libQt6Core_debug.6.dylib (compatibility version 6.0.0, current version 6.5.1)
	/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1971.0.0)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1971.0.0)
	/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 2299.50.120)
	/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 61.0.0)
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1228.0.0)
	/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60420.101.2)
	@rpath/libb2.1.dylib (compatibility version 2.0.0, current version 2.4.0)
	@rpath/libicui18n.73.dylib (compatibility version 73.0.0, current version 73.1.0)
	@rpath/libicuuc.73.dylib (compatibility version 73.0.0, current version 73.1.0)
	@rpath/libicudata.73.dylib (compatibility version 73.0.0, current version 73.1.0)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
	@rpath/libdouble-conversion.3.dylib (compatibility version 3.0.0, current version 3.2.0)
	@rpath/libpcre2-16.0.dylib (compatibility version 12.0.0, current version 12.2.0)
	@rpath/libssl.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	@rpath/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	@rpath/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.5)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1500.65.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

Suggestions applied

[liviaerxin]

Thanks, buddy. Hope it can solve dynamic issues in macOS. I haven't followed up with it for a long time.😂

[BillyONeal]

Pushing a merge to this to end the last uses of the EastAsia fleet (And there were some failures so I didn't view this as resetting things...)

[BillyONeal]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[m-kuhn]

restarting this job as the windows failures were false alerts

[m-kuhn]

@sharadhr thank you for the inputs. I can't look at it before june. If you have something that's working better and ideally takes into account the restructuring approaches, feel free to redo everything and PR against my branch or directly against vcpkg master.

[sharadhr]

@m-kuhn I'm in no hurry!

Personally I'd rather like to keep everything in one place than repeatedly reopening PRs; it makes tracking progress a lot easier. I'm happy to make suggestions and wait for them to be reviewed and accepted when you can. I have a local branch that I can commit to if I really need these changes.

[petersteneteg]

Poke? Any progress on this? Would really appreciate if this was merged!!!

[m-kuhn]

Superseded by https://github.com/microsoft/vcpkg/pull/39313

looking for testers