xamarin-macios icon indicating copy to clipboard operation
xamarin-macios copied to clipboard

Published 20 hours ago • verified publisher icon xamarin

<PublishFolderType> for transitive <Content> files inconsistent behaviour for the non-macos tfm

Open snechaev opened this issue 1 year ago • 6 comments

Steps to Reproduce

  1. Download test.zip
  2. Check the ClassLibrary1.csproj and make sure, that the PublishFolderType set for all content of the Data subfolder 
       <Content Include="Data\**\**">
     <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
       <PublishFolderType>Resource</PublishFolderType>
   </Content>
  3. Build the macOSApp1.csproj
  4. Check the path inside app bundle: /macOSApp1/bin/Debug/net8.0-macos/osx-x64/macOSApp1.app/Contents/Resources/Data/

Expected Behavior

  1. No warnings about the "file does not specify a 'PublishFolderType' metadata" during the build
  2. All the content of the Data folder (1.jpg+2.psd) copied into the app bundle /macOSApp1/bin/Debug/net8.0-macos/osx-x64/macOSApp1.app/Contents/Resources/Data/

Actual Behavior

  1. Build warnings 
    1>Xamarin.Shared.Sdk.targets(1836,3): Warning  : The file '/macOSApp1/ClassLibrary1/Data/2.psd' does not specify a 'PublishFolderType' metadata, and a default value could not be calculated. The file will not be copied to the app bundle.
1>Xamarin.Shared.Sdk.targets(1836,3): Warning  : The file '/macOSApp1/ClassLibrary1/Data/2.psd' does not specify a 'PublishFolderType' metadata, and a default value could not be calculated. The file will not be copied to the app bundle.
  2. Only 1.jpg is copied into the app bundle, the 2.psd is missing.

Environment

Version information (`dotnet --info`) 
.NET SDK:
 Version:           8.0.204
 Commit:            c338c7548c
 Workload version:  8.0.200-manifests.d7126b9e

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  14.5
 OS Platform: Darwin
 RID:         osx-x64
 Base Path:   /usr/local/share/dotnet/sdk/8.0.204/

.NET workloads installed:
 [macos]
   Installation Source: SDK 8.0.200
   Manifest Version:    14.2.8078/8.0.100
   Manifest Path:       /usr/local/share/dotnet/sdk-manifests/8.0.100/microsoft.net.sdk.macos/14.2.8078/WorkloadManifest.json
   Install Type:        FileBased


Host:
  Version:      8.0.4
  Architecture: x64
  Commit:       2d7eea2529

.NET SDKs installed:
  6.0.408 [/usr/local/share/dotnet/sdk]
  6.0.410 [/usr/local/share/dotnet/sdk]
  6.0.412 [/usr/local/share/dotnet/sdk]
  6.0.413 [/usr/local/share/dotnet/sdk]
  6.0.414 [/usr/local/share/dotnet/sdk]
  6.0.415 [/usr/local/share/dotnet/sdk]
  6.0.416 [/usr/local/share/dotnet/sdk]
  6.0.419 [/usr/local/share/dotnet/sdk]
  6.0.420 [/usr/local/share/dotnet/sdk]
  6.0.421 [/usr/local/share/dotnet/sdk]
  6.0.422 [/usr/local/share/dotnet/sdk]
  6.0.424 [/usr/local/share/dotnet/sdk]
  7.0.302 [/usr/local/share/dotnet/sdk]
  7.0.304 [/usr/local/share/dotnet/sdk]
  7.0.306 [/usr/local/share/dotnet/sdk]
  7.0.307 [/usr/local/share/dotnet/sdk]
  7.0.308 [/usr/local/share/dotnet/sdk]
  7.0.309 [/usr/local/share/dotnet/sdk]
  7.0.310 [/usr/local/share/dotnet/sdk]
  7.0.313 [/usr/local/share/dotnet/sdk]
  7.0.314 [/usr/local/share/dotnet/sdk]
  7.0.315 [/usr/local/share/dotnet/sdk]
  7.0.316 [/usr/local/share/dotnet/sdk]
  7.0.317 [/usr/local/share/dotnet/sdk]
  8.0.204 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.16 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.18 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.20 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.21 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.22 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.23 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.24 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.27 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.28 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.29 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.30 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.32 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.10 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.11 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.12 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.13 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.16 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.17 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.18 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.19 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.20 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.16 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.18 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.20 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.21 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.22 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.23 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.24 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.27 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.28 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.29 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.30 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.32 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.9 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.10 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.11 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.12 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.13 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.16 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.17 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.18 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.19 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.20 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Build Logs

msbuild.zip

Example Project (If Possible)

test.zip

Additional information

  • If we will change the tfm for the ClassLibrary1.csproj from the net8.0 to net8.0-macos, then 2.psd will be copied as expected (and build warnings will go away). Multitargeting (net8.0;net8.0-macos) will also work.
  • It looks like that the 1.png is copied (for net8.0 tfm) thanks to the extension-based heuristics.
  • there is also a heuristic check that the item belongs to the <Content/>, but it looks like it does not handle this specific case with 2.psd (although it is a <Content/>) for the net8.0 tfm.

snechaev avatar Jul 26 '24 14:07 snechaev

I did some brief research and here are the results.

  • the PublishFolderType metadata gets lost in the GetCopyToPublishDirectoryItems target in this code. The reason: _GCTPDIKeepMetadata=CopyToPublishDirectory;ExcludeFromSingleFile;TargetPath, so the only the listed metadata are kept and all others (including the PublishFolderType) are dropped.
  • And _GCTPDIKeepMetadata gets its value in this code because the MSBuildDisableGetCopyToPublishDirectoryItemsOptimization is empty for the net8.0 tfm.
  • for the net8.0-macos tfm the MSBuildDisableGetCopyToPublishDirectoryItemsOptimization=true, the optimization is disabled and all the metadata are preserved in the PublishFolderType.

So, looks like that I can just add the

<MSBuildDisableGetCopyToPublishDirectoryItemsOptimization>true</MSBuildDisableGetCopyToPublishDirectoryItemsOptimization>

property in the ClassLibrary1.csproj, but I'm not sure if it is reliable and what side effects it might introduce. At least, it's not very obvious to anyone who will be working with the code as this property is not documented anywhere except in the sdk source comments.

P.S. And I'm still curious anyway if the current behaviour of the "is it is a content file" heuristic check is intended or not.

snechaev avatar Jul 26 '24 18:07 snechaev

So, looks like that I can just add the

<MSBuildDisableGetCopyToPublishDirectoryItemsOptimization>true</MSBuildDisableGetCopyToPublishDirectoryItemsOptimization>

property in the ClassLibrary1.csproj, but I'm not sure if it is reliable and what side effects it might introduce.

As the name indicates, I believe this is just an optimization, so your build might be ever so slightly slower or consume more memory. I don't think this should be an issue unless you have a lot of Content items in your projects though.

P.S. And I'm still curious anyway if the current behaviour of the "is it is a content file" heuristic check is intended or not.

Yes, the behavior and the reasons behind this design is documented here: https://github.com/xamarin/xamarin-macios/blob/main/dotnet/BundleContents.md

The easiest fix is probably multi-target the class library, so that you build for net8.0-macos as well as net8.0.

I'm not entirely sure what we can do to fix this, but I'm leaving this open, because the current behavior is obviously not optimal. One possibility might be to change the definition of _GCTPDIKeepMetadata to include PublishFolderType here: https://github.com/dotnet/sdk/blob/65cd87abc5cedc5cfffbaebe95d643d153f44a92/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Publish.targets#L775.

rolfbjarne avatar Aug 05 '24 18:08 rolfbjarne

From my experience while researching the current behavior after reading the docs, I would say that it might be helpful to emphasize the following moments in the documentation:

  • The "belongs to the content" heuristic does not work for the transitive items, so you need to explicitly set the PublishFolderType for such an items. https://github.com/xamarin/xamarin-macios/blob/bb5cc93d60c7a0c4d5c7b3548d4c802a616d50b1/dotnet/BundleContents.md?plain=1#L48
  • The PublishFolderType metadata does not work for items transitively coming from the non-xamarin projects (targeting net8.0 and not net8.0-macos/ios/etc. I don't know how to formulate this in a simple way in English).
    • The workarounds: manually set MSBuildDisableGetCopyToPublishDirectoryItemsOptimization=true or do a multitargeting net8.0;net8.0-macos/ios/etc for the referenced project.
  • may be to add the example of the warning text to the docs to make it easier to find the docs via Google.

I can make a PR, but my English is far from perfect, so I'm not sure such a PR will be useful (e.g. will not require to full editing/rewriting in the normal English)

snechaev avatar Aug 06 '24 14:08 snechaev

From my experience while researching the current behavior after reading the docs, I would say that it might be helpful to emphasize the following moments in the documentation:

That's a good idea, I'll look into that.

rolfbjarne avatar Aug 06 '24 14:08 rolfbjarne

I'm not entirely sure what we can do to fix this, but I'm leaving this open, because the current behavior is obviously not optimal. One possibility might be to change the definition of _GCTPDIKeepMetadata to include PublishFolderType here: dotnet/sdk@65cd87a/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Publish.targets#L775.

I wonder if we can set MSBuildDisableGetCopyToPublishDirectoryItemsOptimization=true when building dependent projects.

rolfbjarne avatar Oct 14 '24 16:10 rolfbjarne

From my experience while researching the current behavior after reading the docs, I would say that it might be helpful to emphasize the following moments in the documentation:

That's a good idea, I'll look into that.

https://github.com/xamarin/xamarin-macios/pull/21434

rolfbjarne avatar Oct 14 '24 16:10 rolfbjarne