c#: CI is extremely slow

[alexcrichton]

Since C# CI was introduced it's been quite slow, but I realize that I haven't opened an issue about this. CI for C#, specifically on Windows, takes 30+ minutes whereas all other platforms finish in ~10.

For C# folks @jsturtevant, @yowl, @silesmo, would y'all be able to help optimize this? Is there something that can be done to bring the total time closer to the 10 minute range?

[pavelsavara]

I guess the overall C# time is dominated by AOT ? Perhaps we could switch to Mono interpreter for CI ?

https://github.com/bytecodealliance/wit-bindgen/issues/1079

I realize that was not easy with Net9, but Net10 preview1 bits are due in few days. We can try again.

[yowl]

I'm surprised that for these small programs it is quite so slow. Are there logs?

[alexcrichton]

CI logs are from the main.yml workflow. Any green run which ran everything should suffice and the slow builder is "Test (windows-latest, csharp)". The half hour is spent in one step which only has limited logs.

That being said I would expect that running the same command would reproduce this locally pretty easily. If it doesn't then I'd be curious to understand how a test run is so fast on your machine and not on CI!

I can also clarify that at least from my perspective everything about building C# is a large black box to me, so I can't personally answer many questions myself as I mostly just rubber stamped things around CI integration.

[pavelsavara]

It could be also download of the nuget packages Microsoft.DotNet.ILCompiler.LLVM and runtime.win-x64.Microsoft.DotNet.ILCompiler.LLVM. Maybe there is way how to add that to the github action cache ?

[yowl]

I would expect that to only affect the first AOT compiled test (unless we have a nuget.config with a local storage specified). The subsequent tests should then be fast, but there are still many that take over a minute.

[yowl]

unless we have a nuget.config with a local storage specified

We do, trying an alternatvei

[yowl]

Didn't make much difference sharing the nuget location. I would like to see finer timings, I'm not sure how much of each runtime test is rustc, the compiled rust program, the c# compilation, and running the wasm.

[yowl]

Some numbers from testing. I have deleted the target folder then run the strings test followed by the lists test, then repeated without deleting anything. So the first test is going to be slow as it builds everything. Added some rough timing instrumention to main.rs to get these figures, times in seconds. Tested with

cmd /v:on /c "echo !TIME! & cargo test --package wit-bindgen-cli --test runtime --features csharp --no-default-features -- strings::run --exact --nocapture & echo !TIME!

and

cmd /v:on /c "echo !TIME! & cargo test --package wit-bindgen-cli --test runtime --features csharp --no-default-features -- lists::run --exact --nocapture & echo !TIME!

Test	rustc	C#=>Wasm compilation (includes nugets if missing)	Component::from_file	Test Reported	Total
strings1	437	114	34	148	584
lists1	29	13	32	45	74
strings2	27	9	29	39	66
lists2	29	10	32	42	72

Based on these numbers I don't think switching from NAOT to mono is going to save a lot.

[alexcrichton]

With data from https://github.com/bytecodealliance/wit-bindgen/pull/1201 it looks like a major slowdown was Cranelift itself was un-optimized and the C# binaries were big enough to have this be a problem