Support AOT and WebAssembly module trimming

[elringus]

We are currently using a custom .NET runtime build and JS wrapper post-processing to support webpack UMD target. This limits the options when building the projects, such as AOT and module trimming.

There is an on-going work at the .NET runtime main branch to modularize the WASM runtime and make it work outside of browser, eg: https://github.com/dotnet/runtime/pull/61313 https://github.com/dotnet/runtime/pull/62292 https://github.com/dotnet/runtime/issues/47336

At some point, it may become possible to use the module distributed with the runtime SDK, which will resolve the limitations.

If someone happen to know an alternative solution to make current (.NET 6) WASM compatible with UMD target, please let us know here. Sharing updates regarding changes in the .NET runtime that could help with the issue is also appreciated.

[pavelsavara]

@Elringus Here is PR with webpack sample https://github.com/dotnet/runtime/pull/63335 , feedback welcome.

[elringus]

@pavelsavara Hey, Thanks for the info! I've checked the sample webpack config, but it doesn't seem to use UMD library target. It's to make sure the library doesn't depend on any node modules or browser-specific APIs, which is required, for example, by VS Code web extensions. Making the dotnet runtime compatible with the UMD target was the main reason I had to use a custom build. Here is a sample webpack config with UMD target: https://github.com/Elringus/DotNetJS/blob/c6ff664f930cda374a394cce79f642e1e2f8f487/JavaScript/dotnet-runtime/webpack.config.js#L7

When building with that config I had to also fix all the dependency warnings reported by webpack (mostly getting rid of various require and browser API usages in the dotnet.js) to make it work with the VS Code extensions.

[pavelsavara]

I updated the sample to produce UMD, it works fine. No reg-exp fiddling was necessary because I re-define resolve inside the emscripten scope.

[elringus]

That's great news, thank you very much! While I wasn't able to test it due to a build error:

  export EMSDK="C:/Users/Elringus/Desktop/dotnet/src/mono/wasm/emsdk";
  export EM_CONFIG="C:\Users\Elringus\Desktop\dotnet\src\mono\wasm\emsdk\.emscripten";
  export EMSDK_NODE="C:/Users/Elringus/Desktop/dotnet/src/mono/wasm/emsdk/node/14.15.5_64bit/bin/node.exe";
  export EMSDK_PYTHON="C:/Users/Elringus/Desktop/dotnet/src/mono/wasm/emsdk/python/3.9.2-1_64bit/python.exe";
  export JAVA_HOME="C:/Users/Elringus/Desktop/dotnet/src/mono/wasm/emsdk/java/8.152_64bit";
  unset EMSDK_PY;
  [ 25%] Building C object CMakeFiles/dotnet.dir/corebindings.c.o
  system_libs:INFO: retrieving port: zlib from https://github.com/madler/zlib/archive/v1.2.11.zip
  system_libs:INFO: unpacking port: zlib
  cache:INFO: generating port: sysroot\lib\wasm32-emscripten\libz.a... (this will be cached in "C:\Users\Elringus\Desktop\dotnet\emsdk\upstream\emscripten\cache\sysroot\lib\wasm32-emscripten\libz.a" for subsequent builds)
  cache:INFO:  - ok
  [ 50%] Building C object CMakeFiles/dotnet.dir/driver.c.o
  [ 75%] Building C object CMakeFiles/dotnet.dir/pinvoke.c.o
  [100%] Linking C executable dotnet.js
emcc : error : C:UsersElringus.nugetpackagesmicrosoft.netcore.runtime.icu.transport7.0.0-alpha.1.21620.1runtimesbrowser-wasmnativelib/libicuuc.a: No such file or directory ("C:UsersElringus.nugetpackagesmicrosoft.netcore.runtime.icu.transport7.0.0-alpha.1.21620.1runtimesbrowser-wasmnativelib/libicuuc.a" was expected to be an input file, based on the commandline arguments provided) [C:\Users\Elringus\Desktop\dotnet\src\mono\wasm\wasm.proj]
  mingw32-make[2]: *** [CMakeFiles\dotnet.dir\build.make:151: dotnet.js] Error 1
  mingw32-make[1]: *** [CMakeFiles\Makefile2:82: CMakeFiles/dotnet.dir/all] Error 2
  mingw32-make: *** [Makefile:90: all] Error 2
C:\Users\Elringus\Desktop\dotnet\src\mono\wasm\wasm.proj(200,5): error MSB3073: The command "call "C:\Users\Elringus\Desktop\dotnet\eng\native\init-vs-env.cmd" && call "C:\Users\Elringus\Desktop\dotnet\src\mono\wasm\emsdk\emsdk_env.bat" && cmake --build . --config Release" exited with code 2.

Build FAILED.

emcc : error : C:UsersElringus.nugetpackagesmicrosoft.netcore.runtime.icu.transport7.0.0-alpha.1.21620.1runtimesbrowser-wasmnativelib/libicuuc.a: No such file or directory ("C:UsersElringus.nugetpackagesmicrosoft.netcore.runtime.icu.transport7.0.0-alpha.1.21620.1runtimesbrowser-wasmnativelib/libicuuc.a" was expected to be an input file, based on the commandline arguments provided) [C:\Users\Elringus\Desktop\dotnet\src\mono\wasm\wasm.proj]
C:\Users\Elringus\Desktop\dotnet\src\mono\wasm\wasm.proj(200,5): error MSB3073: The command "call "C:\Users\Elringus\Desktop\dotnet\eng\native\init-vs-env.cmd" && call "C:\Users\Elringus\Desktop\dotnet\src\mono\wasm\emsdk\emsdk_env.bat" && cmake --build . --config Release" exited with code 2.
    0 Warning(s)
    2 Error(s)

Time Elapsed 00:21:32.37
Build failed with exit code 1. Check errors above.
Some builds failed:
        Configuration: Release, Architecture: wasm

— if dotnet.js is now packing for UMD without any warnings, the issue is most likely solved. Will check it again when the changes are available in the distributed .NET SDKs (I guess it'll be .NET 7?)

[pavelsavara]

I think there was unrelated zlib hiccup yesterday and the offending PR was reverted since. You just merged in unfortunate moment ...

[elringus]

Well, I've tried one more time by cloning directly from the PR (via gh pr checkout 63335), but the same error occurred. Here is the full log: https://pastebin.com/UtrUNN2v

And here is the script I've used for build (under the repo's root):

mkdir -p emsdk
curl -L https://github.com/emscripten-core/emsdk/archive/2.0.23.tar.gz | tar xz -C emsdk --strip-components=1
emsdk/emsdk update
emsdk/emsdk install 2.0.23
emsdk/emsdk activate 2.0.23
source emsdk/emsdk_env.sh
./build.sh mono+libs -os Browser -c Release
read -r -p "Press Enter key to exit..."

[pavelsavara]

@lambdageek any hints on zlib ?

[pavelsavara]

Also sometimes deleting artifacts and starting clean helps.

[elringus]

I've tried a clean build, debug config (previously I've been using release) and running with elevated command prompt, but the same error occurred in all cases.

Looks like src\mono\wasm\wasm.proj can't resolve path to libicuuc.a (notice the missing slashes):

emcc : error : C:UsersElringus.nugetpackagesmicrosoft.netcore.runtime.icu.transport7.0.0-alpha.1.21620.1runtimesbrowser-wasmnativelib/libicuuc.a: No such file or directory ("C:UsersElringus.nugetpackagesmicrosoft.netcore.runtime.icu.transport7.0.0-alpha.1.21620.1runtimesbrowser-wasmnativelib/libicuuc.a" was expected to be an input file, based on the commandline arguments provided) [C:\Users\Elringus\Desktop\dotnet\src\mono\wasm\wasm.proj]

[lambdageek]

@pavelsavara

get rid of this

https://github.com/dotnet/runtime/blob/e0287b742e51010371b1ab2069df466abbc3fa80/src/mono/wasm/wasm.proj#L85-L88

I'm not sure about the ICU error (although maybe it's related? does ICU depend on zlib?)

[DanielHabenicht]

As long as this does not work, you can use the following workaround to not copy the file from this repo but instead automatically from the nuget package:

    <!-- Because of https://github.com/Elringus/DotNetJS/issues/20 we must use the pre-compiled dotnet.wasm runtime -->
    <ItemGroup>
        <PackageReference Include="DotNetJS" Version="0.9.1" GeneratePathProperty="true"/>
    </ItemGroup>
    <ItemGroup>
        <None Include="$(PkgDotNetJS)\js\dotnet.wasm" TargetPath="wwwroot\_framework\dotnet.wasm" CopyToOutputDirectory="Always" />
        <None Include="$(PkgDotNetJS)\js\dotnet.wasm" TargetPath="dotnet.wasm" CopyToOutputDirectory="Always" />
    </ItemGroup>

[elringus]

It looks like .NET 7 is now compatible with node, but worker/constrained environment support is not clear yet: https://github.com/dotnet/runtime/issues/80045 Though users report its working fine, I'd like to wait until the support is officially confirmed before starting with the migration.