Running unit tests hangs indefinitely

[dmitrynovik]

Running units test hangs indefinitely

1. Clone the repository from https://github.com/dmitrynovik/binary-heap
2. Open the cloned folder with Visual Studio Code (1.41.1)
3. Try to run discovered tests... they hang

The problem is not specific to this code repo but happens to me on other solutions.

The output (note that tests ARE run and the process does terminate; the log .trx file has the output):

Starting extension Finding projects for pattern c:/Users\dmitry.novik\source\repos\Heapsort/ Found 1 matches for pattern in folder c:\Users\dmitry.novik\source\repos\Heapsort Evaluating match c:/Users/dmitry.novik/source/repos/Heapsort/ Adding directory c:/Users/dmitry.novik/source/repos/Heapsort/ Executing dotnet test -t -v=q in c:/Users/dmitry.novik/source/repos/Heapsort/ Executing dotnet vstest "c:\Users\dmitry.novik\source\repos\Heapsort\HeapsortTest\bin\Debug\netcoreapp3.1\HeapsortTest.dll" /ListFullyQualifiedTests /ListTestsTargetPath:"C:\Users\dmitry.novik\AppData\Local\Temp\test-explorer-discover-3em29r\output.txt" in c:/Users/dmitry.novik/source/repos/Heapsort/ Discovered tests with unknown. Found 4 in 1 directories Test run for , expecting 1 test results file(s) in total Executing dotnet build in c:/Users/dmitry.novik/source/repos/Heapsort/ Executing dotnet test --no-build --logger "trx;LogFileName=C:\Users\dmitry.novik\AppData\Local\Temp\test-explorer-y5nvCw\0.trx" in c:/Users/dmitry.novik/source/repos/Heapsort/ Process 45580 started New test results file Process 45580 finished

The answers to the common questions

1. Dotnet sdk version: 3.1.101
2. Unit test framework and relevants versions of that: NUnit (3.12.0)
3. A link to a repo that replicated the issue: https://github.com/dmitrynovik/binary-heap

[mareklinka]

Same issue on the current versions of VSCode (1.43), .NET Core SDK (3.1.102), and the extension (0.7.1).

I have pretty much the same output - the test process finishes, the TRX file is written, but the extension keeps showing the "in progress" spinners:

This was on a partial run - just a single namespace. The test run is rather long (~4 minutes), I ran our integration tests.

[CBielstein]

I'm having the same issue using:

• VS Code 1.43.1
• .NET Core 3.1.200
• .NET Core Test Explorer 0.7.1
• CentOS Linux 7.7.1908

My tests are really short. I was actually trying to play with other extensions and just wrote some really basic tests (one asserts 1 == 1 and the other asserts 2 == 2). Each test project takes < 2 seconds to run, so it's a bit different than the long-running scenario mentioned above.

In case it's helpful, all of my projects are referenced in a Solution file in the top level directory and I'm using default settings on the extension (e.g. I haven't specified a pattern or anything).

Perhaps worth noting, it appears the two unit test files are run separate but both overwrite the same TRX file.

Test run for /home/cameron.bielstein/code/sln-test/test1/bin/Debug/netcoreapp3.1/test1.dll(.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 16.5.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.
Test run for /home/cameron.bielstein/code/sln-test/test2/bin/Debug/netcoreapp3.1/test2.dll(.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 16.5.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.
WARNING: Overwriting results file: /tmp/test-explorer-aOkyR6/0.trx
Results File: /tmp/test-explorer-aOkyR6/0.trx

Test Run Successful.
Total tests: 1
     Passed: 1
 Total time: 1.9738 Seconds
WARNING: Overwriting results file: /tmp/test-explorer-aOkyR6/0.trx
Results File: /tmp/test-explorer-aOkyR6/0.trx

Test Run Successful.
Total tests: 1
     Passed: 1
 Total time: 1.8937 Seconds

At the end, the TRX file looks as follows, which appears to exclude the results for the project test1

<?xml version="1.0" encoding="utf-8"?>
<TestRun id="a23deead-c14a-489f-8a93-501bcd26e454" name="@cameronb 2020-03-27 15:59:54" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2020-03-27T15:59:54.2059679-07:00" queuing="2020-03-27T15:59:54.2059681-07:00" start="2020-03-27T15:59:52.3118323-07:00" finish="2020-03-27T15:59:54.2340537-07:00" />
  <TestSettings name="default" id="17761b2e-48f5-4f6b-8df1-9754c6592cf3">
    <Deployment runDeploymentRoot="_cameronb_2020-03-27_15_59_54" />
  </TestSettings>
  <Results>
    <UnitTestResult executionId="42d6385b-8a33-4dbe-bf4f-7eaf24f4b3b8" testId="febf4f47-57c0-3d5c-9bf5-6bf48298d381" testName="test2.UnitTest1.Test1" computerName="cameronb" duration="00:00:00.0047493" startTime="2020-03-27T15:59:53.9992892-07:00" endTime="2020-03-27T15:59:53.9994381-07:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="42d6385b-8a33-4dbe-bf4f-7eaf24f4b3b8" />
  </Results>
  <TestDefinitions>
    <UnitTest name="test2.UnitTest1.Test1" storage="/home/cameron.bielstein/code/sln-test/test2/bin/debug/netcoreapp3.1/test2.dll" id="febf4f47-57c0-3d5c-9bf5-6bf48298d381">
      <Execution id="42d6385b-8a33-4dbe-bf4f-7eaf24f4b3b8" />
      <TestMethod codeBase="/home/cameron.bielstein/code/sln-test/test2/bin/Debug/netcoreapp3.1/test2.dll" adapterTypeName="executor://xunit/VsTestRunner2/netcoreapp" className="test2.UnitTest1" name="Test1" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="febf4f47-57c0-3d5c-9bf5-6bf48298d381" executionId="42d6385b-8a33-4dbe-bf4f-7eaf24f4b3b8" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
  </TestEntries>
  <TestLists>
    <TestList name="Results Not in a List" id="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Completed">
    <Counters total="1" executed="1" passed="1" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
    <Output>
      <StdOut>[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.0 (64-bit .NET Core 3.1.2)
[xUnit.net 00:00:00.63]   Discovering: test2
[xUnit.net 00:00:00.69]   Discovered:  test2
[xUnit.net 00:00:00.70]   Starting:    test2
[xUnit.net 00:00:00.83]   Finished:    test2
</StdOut>
    </Output>
  </ResultSummary>
</TestRun>

[cavery8989]

I have the same issue as well. Interestingly it was all working really well until I added a second test project. Removing and reinstalling the extension fixed it though.

[kenkendk]

This happens for me too. The process exits, but it looks like the extension deletes the output file. If I run the command manually (while the spinners are showing) it picks up the results and updates the tree. Looks like there is a race of sorts where the result file is deleted before being read, and then it waits for the file forever.

[stefanforsberg]

We've reworked how this is handled in the latest release, curious if you see any improvement?

[mareklinka]

To be honest, in the meantime I forked the code and did some changes and improvements to it, currently running that version. I might give the official version a spin later this week to provide feedback.

Thanks for the update.

[stefanforsberg]

@mareklinka Totally understand, thanks for the update!

Any and all feedback is appreciated and who knows, maybe you can add a PR or two with the improvements you did to you local version sometime in the future? =)

[mareklinka]

I was originally thinking exactly that but it looked like the project might be inactive at the time. I'll definitely give it a look to see what could be useful. I think I'll open a separate issue for that when the time comes.