MacOs bundles are twice the size they should be

[ericstj]

In doing the installer work for ARM64 I noticed our MacOS bundles are twice the size they ought to be. This can be observed by comparing to the tar.gz, or running pkgutil --expand & pkgutil --flatten.

If I run xar -tv -f <file.pkg> on the bundles. I do see evidence of this duplication: https://gist.github.com/ericstj/20eb87cb64e68f98aed3d01de2e08712

I suspect we can do something to cut the size of these files by half.

cc @NikolaMilosavljevic @sfoslund @joeloff

[ericstj]

So I tried this with a simpler installer and can reproduce the behavior. It happens whenever we have two choice elements referencing the same pkg-ref. Even if the pkg-ref has the same ID, with only one entry defining a filename, productbuild will include two copies in the archive that it builds.

This looks to me like a bug in productbuild. What we are doing should be supported per: https://developer.apple.com/library/archive/documentation/DeveloperTools/Reference/DistributionDefinitionRef/Chapters/Distribution_XML_Ref.html#//apple_ref/doc/uid/TP40005370-CH100-SW11

Uniquely identifies a component package within a distribution package—for example, com.apple.TextEdit.pkg. The value of this attribute is usually the same as the component package’s identifier, but it is not required to be the same.

If there are multiple pkg-ref elements with the same ID, their attributes will be collapsed together as if they were specified with a single element. In particular, a pkg-ref element with only an ID can be used inside of a choice element to bind that choice to a pkg-ref that is more fully defined elsewhere in the distribution file.

Content Required. A URL specifying the location of the installation package to which this element refers. Typically, you specify a simple filename and productbuild adjusts the URL as needed when the product archive is created. If you define multiple pkg-ref elements with the same ID, exactly one of them should have text content.

So we follow these rules, only one pkg-ref defines the filename and we use the same pkg-ref id for a given pkg, yet productbuild includes duplicate entries in the pkg (with the exact same filename).

productbuild doesn't seem to be able to use these duplicates, if you expand/flatten the package it continues to work. Also, if you observe the installer log for the product, it refers to the inner content in terms of product.pkg#pkg-ref-filename.pkg which implies it is looking up by the filename which wouldn't be able to deal with the duplicates. I wish this productbuild tool was OSS so I could debug more.

[ericstj]

Here's a repo that demonstrates this issue: https://github.com/ericstj/sample-relocate-product-macos

I also included our attempt at using relocate as well which we couldn't get to work. cc @richlander

[AraHaan]

Oh and this reminds me, is there any reason why on x64 macos that we can install from the arm64 bundles?

[ericstj]

Oh and this reminds me, is there any reason why on x64 macos that we can install from the arm64 bundles?

That seems unusual, mind opening a separate bug in https://github.com/dotnet/installer?

[AraHaan]

Sure, https://github.com/dotnet/installer/issues/12840.

[ericstj]

Transferred this to dotnet/installer for visibility.

[marcpopMSFT]

Old issue triage: @ericstj is this still an issue?

[ericstj]

Seems like this was fixed, perhaps by the apple tools. I see that starting with 3.1.24 / 6.0.4 release the PKG file is back in line with tar.gz. Prior to that (3.1.23 / 6.0.3) the pkg was double the tar.gz in size.