TRX report file folder inconsistent with XML report folder using .runsettings file

[varnk]

• NUnit 3.13.3
• NUnit3TestAdapter 4.2.1
• Visual Studio 2017 Pro 15.9.49
• .NET Framework 4.6.1

When using a .runsettings file and enabling TRX logging, the TRX logs are written relative to the solution dir of the VS project. However, the NUnit XML output is written relative to the NUnit WorkingDir folder. There seems to be no way to have the test result output files go to the same folder with the way the .runsettings file is interpreted by the plug-in.

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <RunConfiguration>
        <ResultsDirectory>.\TestResults</ResultsDirectory>   <!-- This path is relative to Solution Dir -->
    </RunConfiguration>

   <LoggerRunSettings>
       <Loggers>
         <Logger friendlyName="trx" enabled="True">
           <Configuration>
             <LogFileName>TestResults.trx</LogFileName>
           </Configuration>
         </Logger>
       </Loggers>
     </LoggerRunSettings>

    <NUnit>
        <TestOutputXml>.\TestResults</TestOutputXml>            <!-- This path is relative to WorkingDir -->
    </NUnit>
</RunSettings>

[OsirisTerje]

@varnk Correct!
The TRX logging is handled by the testhost, which know where the solution directory is, and can point to it using a relative path. The test adapter only knows about the working directory, and have no information about where the solution directory is, so the output folder is relative to that working directory.

The adapter doesn't read the runsettings itself either - otherwise it could have used that file's location as a clue - instead the runsettings is passed down to the adapter from the testhost.

We have asked for such metadata to be present somewhere, e.g. runsettings structure, but there are none now. The adapter depends on the information it gets from the testhost.

That said, if you in your solutions establish a fixed convention for where you have placed your test projects, then you can use that information to point upwards to the solution directory and thus the TestResults folder.

[OsirisTerje]

I had to have another peak inside what we get from the testhost, and now I do see we actually get the solution directory and the test result directory in the runcontext (See TestRunDirectory). I am pretty sure that it was not present last time I checked, but I can ofc be wrong about that too.
I also see that the runsettings setting for the result directory is expanded out to an absolute path.

So, this issue is actually fixable.

In order to not break those who rely on the current folder, it will probably need a feature flag to switch behavior.

Thanks @varnk for making us aware of this!