Significant build time and bundle size for .NET MAUI on iOS

[jenyangk]

Description

As of today, my builds for my .NET MAUI application on iOS is taking significantly longer to build, from 10 minutes to 90 minutes. The bundle size has also doubled from 123 MB to 233 MB.

My team and I aren't sure what changed between today's and yesterdays' macOS-13 image that would affect this. But, something definitely changed between today and yesterday. We ran a few builds yesterday and didn't encounter anything wrong.

We've tried setting the Xcode version to 15.2 and .NET SDK version from 8.0.x (which had defaulted to 8.0.303 on the 9th of July) to 8.0.302 (which was the 8.0.x version used on the 8th of July).

The build time and bundle size for Android is the same. No change there. Only affecting iOS builds.

Platforms affected

• [X] Azure DevOps
• [ ] GitHub Actions - Standard Runners
• [ ] GitHub Actions - Larger Runners

Runner images affected

• [ ] Ubuntu 20.04
• [ ] Ubuntu 22.04
• [ ] Ubuntu 24.04
• [ ] macOS 11
• [ ] macOS 12
• [X] macOS 13
• [ ] macOS 13 Arm64
• [ ] macOS 14
• [ ] macOS 14 Arm64
• [ ] Windows Server 2019
• [ ] Windows Server 2022

Image version and build link

20240707.2. Sorry, no public builds link.

Is it regression?

20240630.1

Expected behavior

Build should finish within 15 minutes with a 120+ MB bundle size.

Actual behavior

The build is taking 90 minutes to finish and builds a bundle that is double in size.

Repro steps

Run build script on Azure Pipelines with macOS-13 as the image.

- task: Bash@3
   displayName: Build iOS App
   inputs:
      targetType: 'inline'
      script: |
            cd $(ProjectLocation)
            dotnet publish -f net8.0-ios -c:$(BuildConfiguration) /p:CodesignKey='$(CodesignKey)' /p:CodesignProvision='$(CodesignProvision)'

[jenyangk]

I went back an re-ran a few old builds of mine from last week which were running normally (finishing within 10 - 20 minutes). They are also now taking longer to finish (~ 1 hour 30 minutes). Definitely not a change to our configuration or code.

[pierre-galaup]

Hello,

I have the same problem with my runner for MAUI iOS too. I tried to update to a Macos-14 and xcode 15.4 but it doesn't change anything.

[sureshe456]

Hi @jenyangk, We are looking into it. Will keep you posted.

[IeuanWalker]

Not sure if this helps the investigation or not.

But we have just built a pipeline for .net maui iOS and was shocked at the build times, but we are using Azure pipelines, so not sure if it's using the same infrastructure behind the scenes.

[ESO-ST]

I'm experiencing the same issue, this started happening with image version '20240708.1' builds used to take 20 minutes, now they're hitting our timeout limit at 60 minutes, I'll increase our timeout to see if they'll actually complete.

Agent name: 'Hosted Agent'
Agent machine name: 'Mac-1720751551587'
Current agent version: '3.241.0'
Operating System
macOS
14.5
23F79
Runner Image
Image: macos-14
Version: 20240708.1
Included Software: https://github.com/actions/runner-images/blob/macos-14/20240708.1/images/macos/macos-14-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-14%2F20240708.1
Runner Image Provisioner
2.0.370.1+55cddbb57f254a369b6e7b3b508cebdbee5d24be
Current image version: '20240708.1'

[giorgitedi]

We have been facing the same issue since July 9th. Previously, the building time was about 20 minutes, but now it takes 1 hour and 30 minutes. Additionally, the app size has increased from 90MB to 120MB. Most importantly, the app has started crashing.

We haven't made any changes or added new files, except for some functional/logical updates in the code base.

[ronan-burke-civ]

Most importantly, the app has started crashing.

I'm seeing the exact same thing. Running the pipeline on an old commit from a few weeks ago now produces a build that crashes on startup. I'm pinning .NET MAUI version and .NET version in my pipeline. The only thing I can think of that has changed is the hosted DevOps runner?

[giorgitedi]

Most importantly, the app has started crashing.

I'm seeing the exact same thing. Running the pipeline on an old commit from a few weeks ago now produces a build that crashes on startup. I'm pinning .NET MAUI version and .NET version in my pipeline. The only thing I can think of that has changed is the hosted DevOps runner?

I found this comment from last week and I think it is related.

Can you share a build task part from .yml file?

[ronan-burke-civ]

I'm pinning Xcode to 15.2 in the pipeline already so I'm not sure if it is that. This is how I'm building the app:

  - task: DotNetCoreCLI@2
    displayName: 'Build iOS App'
    inputs:
      command: 'build'
      projects: '$(singleProjectCsproj)'
      arguments: '-c Release -f net8.0-ios -o "$(outputDirectory)" /p:BuildIpa=true /p:ArchiveOnBuild=true /p:RuntimeIdentifier="ios-arm64" /p:CodesignKey="$(APPLE_CERTIFICATE_SIGNING_IDENTITY)" /p:CodesignProvision=$(APPLE_PROV_PROFILE_UUID)'

MacOS 13 + Xcode 15.2 doesn't work.

Current agent version: '3.241.0'
Operating System
macOS
13.6.7
22G810
Runner Image
Image: macos-13
Version: 20240707.2
Included Software: https://github.com/actions/runner-images/blob/macos-13/20240707.2/images/macos/macos-13-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-13%2F20240707.2
Runner Image Provisioner
2.0.370.1+55cddbb57f254a369b6e7b3b508cebdbee5d24be
Current image version: '20240707.2'

MacOS 14 + Xcoce 15.3 also doesn't work (only tried it today so I've not had a historical build working with this combo)

---

MacOS 13 + Xcode 15.2 previously did work:

Current agent version: '3.241.0'
Operating System
Runner Image
Image: macos-13
Version: 20240616.1
Included Software: https://github.com/actions/runner-images/blob/macos-13/20240616.1/images/macos/macos-13-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-13%2F20240616.1
Runner Image Provisioner
2.0.370.1+55cddbb57f254a369b6e7b3b508cebdbee5d24be
Current image version: '20240616.1'

[giorgitedi]

@ronan-burke-civ Managed to reduce build time with this but now some of the functionality doesn't work and app crashes again.

[ESO-ST]

This is the last image version that worked

Current agent version: '3.241.0'
Operating System
macOS
14.5
23F79
Runner Image
Image: macos-14
Version: 20240701.1
Included Software: https://github.com/actions/runner-images/blob/macos-14/20240701.1/images/macos/macos-14-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-14%2F20240701.1
Runner Image Provisioner
Current image version: '20240701.1'
Agent running as: 'runner'

and after the upgrade to Version: 20240708.1 the build time went from 20 minutes to 1h20m and the app crashes on startup. nothing else changed, this is building the same code, just different image version. @sureshe456 Can we please revert back to 20240701.1 as you work on this, because at the moment we are completely blocked.

Current agent version: '3.241.0'
Operating System
macOS
14.5
23F79
Runner Image
Image: macos-14
Version: 20240708.1
Included Software: https://github.com/actions/runner-images/blob/macos-14/20240708.1/images/macos/macos-14-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-14%2F20240708.1
Runner Image Provisioner
Current image version: '20240708.1'
Agent running as: 'runner'

[ESO-ST]

is there a way we can tell the pipeline exactly which image version to use?

[jenyangk]

is there a way we can tell the pipeline exactly which image version to use?

@ESO-ST For Microsoft-hosted agents, I don't believe there's a way to explicitly set the build agent version to use.

If you were using a self-hosted agent though, you could probably set it.

[crhalvorson]

We were experiencing this issue, but pinning to a specific MAUI version (in our case 8.0.40) appears to have resolved it for us. May not work for everyone. We were previously just doing a dotnet workload install without rollback file.

dotnet workload update --from-rollback-file workload-rollback-file.json 
dotnet workload install android ios maui --skip-manifest-update

We created a rollback json by using the command dotnet workload update --print-rollback in a local working environment.

That said, looking back at pipeline logs - the first failed build with this symptom occurred with MAUI 8.0.40, but all subsequent ones were 8.0.61 which has some reported issues

[ESO-ST]

We were experiencing this issue, but pinning to a specific MAUI version (in our case 8.0.40) appears to have resolved it for us. May not work for everyone. We were previously just doing a dotnet workload install without rollback file.

dotnet workload update --from-rollback-file workload-rollback-file.json 
dotnet workload install android ios maui --skip-manifest-update

We created a rollback json by using the command dotnet workload update --print-rollback in a local working environment.

That said, looking back at pipeline logs - the first failed build with this symptom occurred with MAUI 8.0.40, but all subsequent ones were 8.0.61 which has some reported issues

building with MAUI 8.0.40 does not fix any of the issues for us, build times still very long and app crashes on startup.

[giorgitedi]

Any updates from the official side? We are unable to release the new version of the app.

[susmitamane]

@giorgitedi still we are investigating the issue and will keep you posted.

[ESO-ST]

@giorgitedi still we are investigating the issue and will keep you posted.

any chance of reverting the image back to one that worked (20240701.1) while this is being worked on? we've been blocked for a week now :(

[giorgitedi]

We were experiencing this issue, but pinning to a specific MAUI version (in our case 8.0.40) appears to have resolved it for us. May not work for everyone. We were previously just doing a dotnet workload install without rollback file.

dotnet workload update --from-rollback-file workload-rollback-file.json 
dotnet workload install android ios maui --skip-manifest-update

We created a rollback json by using the command dotnet workload update --print-rollback in a local working environment. That said, looking back at pipeline logs - the first failed build with this symptom occurred with MAUI 8.0.40, but all subsequent ones were 8.0.61 which has some reported issues

building with MAUI 8.0.40 does not fix any of the issues for us, build times still very long and app crashes on startup.

@ESO-ST This approach worked for us, and the app looks stable again. Could you let me know if you tried this command exactly?

dotnet workload install maui --from-rollback-file https://maui.blob.core.windows.net/metadata/rollbacks/8.0.40.json

[crhalvorson]

Looking at the diff of packages included in our first failed vs. last good build logs again, I think the issue may have been with one of these prerequisites rather than MAUI 8.0.40 (although the official rollback file should get you there since it pins these also). MAUI 8.0.40 was used in both of these. Our working pinned json file includes the "last working" versions here.

last working:

Installing workload manifest microsoft.net.sdk.ios version 17.2.8053...
Installing workload manifest microsoft.net.sdk.maccatalyst version 17.2.8053...
Installing workload manifest microsoft.net.sdk.macos version 14.2.8053...
Installing workload manifest microsoft.net.sdk.maui version 8.0.40...
Installing workload manifest microsoft.net.sdk.tvos version 17.2.8053...

First failing:

Installing workload manifest microsoft.net.sdk.ios version 17.2.8078...
Installing workload manifest microsoft.net.sdk.maccatalyst version 17.2.8078...
Installing workload manifest microsoft.net.sdk.macos version 14.2.8078...
Installing workload manifest microsoft.net.sdk.maui version 8.0.40...
Installing workload manifest microsoft.net.sdk.tvos version 17.2.8078...

[ESO-ST]

We were experiencing this issue, but pinning to a specific MAUI version (in our case 8.0.40) appears to have resolved it for us. May not work for everyone. We were previously just doing a dotnet workload install without rollback file.

dotnet workload update --from-rollback-file workload-rollback-file.json 
dotnet workload install android ios maui --skip-manifest-update

We created a rollback json by using the command dotnet workload update --print-rollback in a local working environment. That said, looking back at pipeline logs - the first failed build with this symptom occurred with MAUI 8.0.40, but all subsequent ones were 8.0.61 which has some reported issues

building with MAUI 8.0.40 does not fix any of the issues for us, build times still very long and app crashes on startup.

@ESO-ST This approach worked for us, and the app looks stable again. Could you let me know if you tried this command exactly?

dotnet workload install maui --from-rollback-file https://maui.blob.core.windows.net/metadata/rollbacks/8.0.40.json

you are correct, using that command works. I had misunderstood your original message, thinking you meant to build using 8.0.40 MAUI packages (nuget).

I don't think using this command as a workaround will be an issue since our project references the 8.0.70 MAUI packages, so it will still be building using those.

[susmitamane]

Hi @giorgitedi We tried building few sample projects for .NET with mac-os 13,14 as you have suggested in the repro steps, but it seems like current image versions doesn't have that issue. while testing both sample project i am not seeing any issue regarding build. Build is completing in required time for both of the tested applications.

Sample Projects:

1. https://github.com/susmitamane/MauiReproEmit/actions/runs/9992299697/job/27616970774
2. https://github.com/susmitamane/waf/actions/runs/9976548782/job/27569021051

In your case application bundle size may vary. could you please try with mac-os updated images, and feel free to get back to us with results.

Thanks.

[giorgitedi]

@susmitamane I tried again today without "--from-rollback-file " parameter and still didn't work.

[Redth]

Can someone try building with the 8.0.40 workloads (using the rollback file mentioned) but having their project(s) reference the 8.0.61 NuGet package? I suspect this isn't related to any MAUI changes specifically, but more likely something within the iOS workloads. It would be great to have a data point to (in)validate this assertion.

If you're able to share binlog's for each scenario (slow vs older faster builds) that would also be helpful.

[Redth]

May be related: https://github.com/xamarin/xamarin-macios/issues/20848

[ESO-ST]

Can someone try building with the 8.0.40 workloads (using the rollback file mentioned) but having their project(s) reference the 8.0.61 NuGet package? I suspect this isn't related to any MAUI changes specifically, but more likely something within the iOS workloads. It would be great to have a data point to (in)validate this assertion.

If you're able to share binlog's for each scenario (slow vs older faster builds) that would also be helpful.

I can confirm that assertion is correct, I'm building using the rollback file mentioned but my project is referencing MAUI 8.0.70 nuget packages, and this workaround is functional.

I have the build logs for the two scenarios, no workaround slow builds and app crashes on startup vs workaround everything is normal, but I can't attach them publicly, is there a way I can privately share them?

[susmitamane]

@giorgitedi As explained earlier we tried reproducing issue with mac-os latest image for multiple sample projects but not seen mentioned build related issue. I think the issue is not related to mac-os images but related to the .NET framework. as you can see the thread mentioned by user @Redth(https://github.com/xamarin/xamarin-macios/issues/20848) analysed it and as mentioned it is more on side of xamrin or .NET framework(MAUI) issues. there is another thread going on with area .NET you may find answer to your questions there.

Reference comment:

• https://github.com/xamarin/xamarin-macios/issues/20848#issuecomment-2220644472.
• https://github.com/xamarin/xamarin-macios/issues/20848#issuecomment-2225597944

As mentioned in the comment it is more related to workload and not related images.

• Workaround Suggested: https://github.com/xamarin/xamarin-macios/issues/20848#issuecomment-2225722575

Marking this issue as external as it is not related to macos images and closing it.Feel free to get back to us for mac-os related issues. Thank you.