testfx icon indicating copy to clipboard operation
testfx copied to clipboard

Single file application slow on osx-x64 runtime

Open thomhurst opened this issue 1 year ago • 3 comments

I originally thought this was a dotnet runtime issue but a single file executable basic console app is quicker on all runtimes than invoking dotnet run.

However once you add the Microsoft.Testing.Platform then a osx-x64 single file executable takes a lot longer.

Benchmarks are here: https://github.com/thomhurst/SlowOsx64SingleFile/actions/runs/10811251824

Basic reproduction is available in that repository too.

thomhurst avatar Sep 11 '24 12:09 thomhurst

Do you know if this reproes only on M1 or also on Intel chip on MacOS?

nohwnd avatar Sep 11 '24 12:09 nohwnd

Do you know if this reproes only on M1 or also on Intel chip on MacOS?

Don't know off the top of my head. Whatever the GitHub actions macos-latest uses

thomhurst avatar Sep 11 '24 12:09 thomhurst

the benchmark you shared run on M1. Thanks.

nohwnd avatar Sep 11 '24 12:09 nohwnd

Image still observable.

nohwnd avatar Feb 24 '25 15:02 nohwnd

Interesting, I am wondering if we should ping some people from runtime so we could maybe have some pointers on what could have such big impact.

Evangelink avatar Feb 24 '25 19:02 Evangelink

That was the original thought, but it only reproes when our packages are added, so I guess we need some traces at least, before bothering runtime people, postponed till my wife lends me her Mac 😁

nohwnd avatar Feb 25 '25 13:02 nohwnd

Any plans to address this? You can see in TUnit, at the bottom of the readme, I've got benchmarks, and all are slower on Mac

thomhurst avatar Sep 25 '25 18:09 thomhurst

No specific plans yet, it has been opened on my computer for a while, but I don't have a mac to debug this on, so I will try to get traces when running on hosted machine and see if there is something to see.

nohwnd avatar Sep 30 '25 10:09 nohwnd

I tried it and it seems that the benchmark is incorrect. The benchmark compares dotnet run, which is running natively on arm64, with the single file app that is built for non-native x64, and hence is virtualized and slower.

nohwnd avatar Oct 01 '25 10:10 nohwnd

https://github.com/thomhurst/SlowOsx64SingleFile/pull/1

nohwnd avatar Oct 01 '25 10:10 nohwnd

Image

This is what I see when I fix the build to use osx-arm64. Comparable data with some small overhead probably caused by the singlefileappness.

nohwnd avatar Oct 01 '25 10:10 nohwnd

https://github.com/thomhurst/TUnit/blob/main/run-singlefile-tests.ps1#L27 same thing happening here, you probably want to detect the architecture. Or use arm64 for macos if you use the latest images, because they are arm64 https://github.com/actions/runner-images?tab=readme-ov-file#available-images

nohwnd avatar Oct 01 '25 11:10 nohwnd

Results from run on native:

Image

nohwnd avatar Oct 01 '25 11:10 nohwnd

Doh! It's always something so simple isn't it?

Thanks for finding that!

thomhurst avatar Oct 01 '25 11:10 thomhurst

If only it was always this simple :)

nohwnd avatar Oct 01 '25 12:10 nohwnd