BenchmarkDotNet
BenchmarkDotNet copied to clipboard
Consider adding tests for WASM support
WASM support got again broken very recently: https://github.com/dotnet/runtime/issues/78575
We should consider adding automated tests for verifying that WASM just works.
Our Linux CI jobs takes 20+ minutes to execute, so there is plenty of time we can consume (the limit is 1h).
We should mimic what we do for NativeAOT:
- Install dependencies on Linux: https://github.com/dotnet/BenchmarkDotNet/blob/ad8e9b26ef22aa77abff2469f7668986e1449127/azure-pipelines.Ubuntu.yml#L17-L22 https://github.com/dotnet/BenchmarkDotNet/blob/ad8e9b26ef22aa77abff2469f7668986e1449127/.github/workflows/build.yaml#L26-L27
- Run against hardcoded version to avoid WASM breaking us.
Ideally, we should update the SDK to .NET 7 first: https://github.com/dotnet/BenchmarkDotNet/issues/2080
We don't need to build entire monoVM as described in https://benchmarkdotnet.org/articles/configs/toolchains.html#wasm. We just need to install v8 as described in https://github.com/dotnet/runtime/blob/main/docs/workflow/testing/libraries/testing-wasm.md and follow steps described by @radical here: https://github.com/dotnet/BenchmarkDotNet/issues/1818#issuecomment-1208443917
cc @radical @naricc
I think wasm tests here can avoid building the latest dotnet/runtime as you mentioned:
- use latest dotnet 7.0
dotnet workload install wasm-tools --skip-manifest-update- this would allow running the tests with relinking, and AOT.- install
v8 - run some tests!
Essentially, the tests here will provide a basic sanity check that wasm apps can be benchmarked. In addition, the dotnet/performance repo should have similar tests but those will run some microbenchmarks from dotnet/runtime as a sanity check. And that can follow the sdk version being used by dotnet/runtime for testing.
@radical Do we need to rework how the wasm toolchain works? Or do we already support running it against whatever is in the the workload?
If you are asking whether running against the workload needs to be implemented, then no, it already works, and dotnet/runtime+dotnet/performance use that.