BenchmarkDotNet icon indicating copy to clipboard operation
BenchmarkDotNet copied to clipboard

improve wasm support

Open FreyLuis opened this issue 4 weeks ago • 4 comments

I found a few issues while setting up WASM benchmarks & would like to fix those. Since those would include breaking changes, I’m asking about them here before starting any work.

  • Currently --expose_wasm is the default parameter for javaScriptEngineArguments, but the flag was removed in 2024.
  • The JS entry point is hardcoded to be AppBundle/test-main.js (or AppBundle/main.js with .net 5 / 6). The user then has to create the file under the correct path & still needs to set the WasmMainJSPath as an msbuild argument. The path shouldn’t be hardcoded & BenchmarkDotNet could autogenerate the file.
  • The default parameter for the .net monikers are still .net 5. IMHO, those should either be removed or replaced with factory methods (CreateWasm10_0(string javaScriptEngine, string javaScriptEngineArguments, bool aot)).

The docs are also outdated (already tracked by https://github.com/dotnet/BenchmarkDotNet/issues/1818):

  • The current example only describes how to set benchmarks up using a custom dotnet runtime. This is unnecessary and looks scary to anyone not working on dotnet itself.
  • The example also recommends using v8 directly via jsvu, which does not provide standard JavaScript APIs. Dotnet relies on these APIs (for creating GUID’s, it uses crypto.getRandomValues() for example). IMHO, recommending node might be the better choice.

Would you accept such a PR, or is someone already working on something similar?

FreyLuis avatar Dec 02 '25 17:12 FreyLuis

  • The default parameter for the .net monikers are still .net 5. IMHO, those should either be removed or replaced with factory methods (CreateWasm10_0(string javaScriptEngine, string javaScriptEngineArguments, bool aot)).

I have mentioned in the past that the default moniker should probably use the current runtime version, but no one seemed interested in doing it.

We are definitely open to accepting PRs that update the wasm toolchain. Just make sure you validate any changes against dotnet/performance.

timcassell avatar Dec 02 '25 22:12 timcassell

That probably explains my issue in https://github.com/dotnet/BenchmarkDotNet/discussions/2675. Maybe we can enable Windows support with your updates.

timcassell avatar Dec 05 '25 07:12 timcassell

@timcassell Sure, I was already wondering what blocks windows support. I believe that the run-v8.sh script generated with the WasmGenerateRunV8Script MsBuild Property also contains the flag, but BenchmarkDotNet doesn't use the script, so this isn't an issue here.

Would it be fine to completely remove the wasmDataDir command line argument? I believe it's not actually an option in the WebAssembly SDK and just used here in dotnet/performance to set the main.js path. Instead, we could add a wasmMainJsPath option.

FreyLuis avatar Dec 05 '25 13:12 FreyLuis

It sounds fine to me, but I'm no wasm expert.

timcassell avatar Dec 08 '25 00:12 timcassell

  • The default parameter for the .net monikers are still .net 5. IMHO, those should either be removed or replaced with factory methods (CreateWasm10_0(string javaScriptEngine, string javaScriptEngineArguments, bool aot)).

After a bit more thought, I think they should just be removed. All the other monikers (except old mono) are version-specific.

timcassell avatar Dec 14 '25 11:12 timcassell