Running on Android and iOS devices with net6.0

[RcBrgr]

Hi there,

with Xamarin it is possible to run Benchmarks directly on the Android or iOS device like also shown in the Xamarin.Forms sample in this repo. Unfortunately that is not possible anymore with dotnet maui with the project targeting net6.0-android and/or net6.0-ios. When running Benchmarks on Android the System.ArgumentOutOfRangeException: "Runtime not supported (Parameter 'runtime') Actual value was MonoAOTLLVM." will get thrown. Same happens on iOS. Are there any workarounds?

[adamsitnik]

Hi @RcBrgr

Could you please provide a full stack?

BDN is being used in maui repo and I would expect to work fine:

https://github.com/dotnet/maui/blob/main/src/Core/tests/Benchmarks.Droid/MainInstrumentation.cs

[RcBrgr]

Hi @adamsitnik, sorry for my late response. First things first providing a custom logger implementation and modifying the config like in the link you have provided, benchmarks are running on the device in a Maui application. However some workarounds are needed like described in the file you have linked.

Coming to the implementation like in the sample for Xamarin Forms.

// BenchmarkComponent.cs
...
IsRunning = true;

            try
            {
                var logger = new AccumulationLogger();
                await Task.Run(() =>
                {
                    var config = default(IConfig);
#if DEBUG
                    config = new DebugInProcessConfig();
#endif
line 58:        var summary = BenchmarkRunner.Run(_benchmarkTypes.FirstOrDefault(t => t.Name == SelectedBenchmarkName), config);
                    MarkdownExporter.Console.ExportToLog(summary, logger);
                    ConclusionHelper.Print(
                        logger,
                        summary.BenchmarksCases
                            .SelectMany(benchmark => benchmark.Config.GetCompositeAnalyser().Analyse(summary))
                            .Distinct()
                            .ToList());
                });
                Summary = logger.GetLog();
            }
            catch (Exception exc)
            {
                await Application.Current.MainPage.DisplayAlert("Error", exc.Message, "Ok");
            }
            finally
            {
                IsRunning = false;
            }

There is no special config for the Benchmark itself in place or something like that. This is working fine with Xamarin Forms. This is the Stack when running this code in a Maui application:

System.PlatformNotSupportedException
Operation is not supported on this platform.
System.Console
   at System.ConsolePal.set_Title(String value)
   at System.Console.set_Title(String value)
   at BenchmarkDotNet.Running.ConsoleTitler..ctor(String initialTitle)

Closed because it is working with the workarounds in the following link: https://github.com/dotnet/maui/blob/main/src/Core/tests/Benchmarks.Droid/MainInstrumentation.cs

[adamsitnik]

However some workarounds are needed

Then we should reopen this issue and fix it.

The stack trace that you have provided is very unexpected, as the access to Console.Title is guarded with catch block that swallows PlatformNotSupportedException:

https://github.com/dotnet/BenchmarkDotNet/blob/0c26996ea685a99068aca71e7ae547b0851d3c64/src/BenchmarkDotNet/Running/ConsoleTitler.cs#L40-L52

@jonathanpeppers is it possible that we can't catch PlatformNotSupportedException on Maui? Or that the exception stack points to the Title setter but it's the getter that throws (the call to getter is not catching PNSE)?

[jonathanpeppers]

The OP is using .NET 6, but it seems like it should still work:

https://github.com/dotnet/runtime/blob/release/6.0/src/libraries/System.Console/src/System/ConsolePal.Android.cs

You should be able to catch System.PlatformNotSupportedException, so I'm not sure why it's not working.

@RcBrgr any different results with .NET 7?

[RcBrgr]

@jonathanpeppers with targeting .NET 7 it is working with this code. Thank you very much.

[themronion]

Quick question here, is xamarin still supported? Cause i get the same exception when i run the sample

[jonathanpeppers]

Xamarin was working for me at one point: https://github.com/jonathanpeppers/BenchmarkDotNet-Android

.NET 6+ and MAUI should for sure work, I believe we have some benchmarks running in a CI pipeline somewhere.

[themronion]

Yeap, downgrading helped to launch it on Xamarin

[BraedenLS]

This issue remains. I believe (in my case anyway) it incorrectly determines that I am running with the MonoAOTLLVM runtime, when I am just using Mono. If this is not the problem, then a Toolchain for MonoAOTLLVM needs to be created and returned in the GetToolchain() method. The current workaround is to manually set your benchmark class as a [MonoJob]

[jonathanpeppers]

This project is working in .NET 9:

• https://github.com/dotnet/maui/blob/main/src/Core/tests/Benchmarks.Droid/Benchmarks.Droid.csproj

There might be some hacks in there that make things work, but it's been a while, so not remembering fully.