ci: enable testing in true isolation

[mihai-unity]

We are experiencing a hang in our repo where we are building API Validation assemblies in which we take your latest promoted package [email protected] (not happening with 3.1.0-exp.3). The script imports the package with the Windows Unity Editor for the version you have specified in the unity field of your package (i.e. 2020.3).

We think it's because the use of SynchronizationContext are not properly cleaned up after and Unity still waits for them. Unfortunately this doesn't reproduce when we run the same commands on the command line but this PR should, in theory, reproduce the same hang on your repo as well.

Enabling testing in true isolation is considered a good practice since this way you should be able to spot also missing dependencies since we are loading the package in a empty project and it should compile without errors.

We were also wondering the reason you added that huge --timeout arg to utr if it's related to this issue but we were unable to find any job that exceeded reasonable timings.

[mihai-unity]

@karasusan can you take a quick look at this?

[karasusan]

@mihai-unity Thank you for the PR. I am checking. I've never known the command option. Can I have a project which you experiencing a hang?

[mihai-unity]

@karasusan that's the problem, reproducibility. The repro project should be as simple as an empty project with default packages plus your package and then you open the editor with -batchmode -quit and it should hang. The main problem why we haven't figured out why yet, is that it only happens in Yamato remote for whatever reason. That's why my attempt was to try reproduce it in a CI job in your repo so that you could have some way of telling you've fixed it.

You could even login to the VM and debug Unity right there and see what it is waiting on but I have no idea why it doesn't reproduce if you would run the same commands locally.

[mihai-unity]

and oh, I just noticed that you are running the tests with a GPU attached. The test should reproduce using a simple Unity::VM Windows VM.

[karasusan]

and oh, I just noticed that you are running the tests with a GPU attached. The test should reproduce using a simple Unity::VM Windows VM.

Got it. I will add the job for testing without GPU.

[mihai-unity]

But you should notice already some errors like this:

Library\PackageCache\[email protected]\Runtime\Scripts\AudioCustomFilter.cs(15,30): error CS1069: The type name 'AudioSource' could not be found in the namespace 'UnityEngine'. This type has been forwarded to assembly 'UnityEngine.AudioModule, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' Consider 

coming from com.unity.webrtc which fortunately you own that as well. So in order to fix that error you'd first need to make that package work in true isolation.

[mihai-unity]

Let's punt on that since I think it would be a different effort. Let's try something else.

[mihai-unity]

can you cheery-pick 2583fbf5f484f734eb316c8758dbd38c96233ffd ?

[karasusan]

I reproduced the issue on CI. I am checking the behavior when working on Windows without GPU.

[karasusan]

I am fixing this error here. We need to add the dependency com.unity.module.audio package for the WebRTC package.

Library\PackageCache\[email protected]\Runtime\Scripts\AudioCustomFilter.cs(15,30): error CS1069: The type name 'AudioSource' could not be found in the namespace 'UnityEngine'. This type has been forwarded to assembly 'UnityEngine.AudioModule, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' Consider 

[karasusan]

This new PR #806 contains #796 changes. I will close this issue. Thank you.