maui icon indicating copy to clipboard operation
maui copied to clipboard

[iOS][NativeAOT] Stand up testing on devices

Open ivanpovazan opened this issue 1 year ago • 1 comments

Description

With https://github.com/dotnet/maui/pull/19194 we started testing NativeAOT iOS support with MAUI on CI. However, added tests are only testing building MAUI iOS template applications. In order to have a better test coverage we should stand up running NativeAOT iOS tests on devices as well.

Tasks

This effort should cover at least the following:

  • [ ] Stand up running RunOniOS tests with NativeAOT https://github.com/dotnet/maui/blob/f9a885219953dd67171b0986f9746795fc5fc207/src/TestUtils/src/Microsoft.Maui.IntegrationTests/AppleTemplateTests.cs#L36
  • [ ] Stand up running Core.DeviceTests.csproj tests with NativeAOT https://github.com/dotnet/maui/blob/main/src/Core/tests/DeviceTests/Core.DeviceTests.csproj
  • [ ] TBD - Additional relevant testing

ivanpovazan avatar Jan 11 '24 08:01 ivanpovazan

/cc: @jonathanpeppers @simonrozsival We should discuss what will fall into TBD - Additional relevant testing but feel free to extend the list as you see fit.

ivanpovazan avatar Jan 11 '24 08:01 ivanpovazan

Regarding:

Stand up running Core.DeviceTests.csproj tests with NativeAOT

In order to adapt these tests to execute with NativeAOT, we will need to adapt/change HeadlessTestRunner https://github.com/dotnet/maui/blob/c20f10de068e806158fa4436da3855e6bb9fc9d1/src/TestUtils/src/DeviceTests.Runners/HeadlessRunner/iOS/HeadlessTestRunner.cs#L13 as it inherits XHarness.TestRunners.Xunit test runner. The problem with this type of runners, is that it expects assemblies to be present on disk in order to load them and find relevant test cases. This is not supported by NativeAOT as the managed assemblies are not preserved during deployment.

  • As a short-term solution/workaround, we could potentially create a NativeAOT variant of HeadlessTestRunner which resembles single file test runner (similar we are using in dotnet/runtime)
  • As a long-term solution, we should create an AOT-compatible XUnit test runner: https://github.com/dotnet/runtime/issues/91069

ivanpovazan avatar Jan 17 '24 11:01 ivanpovazan