Several problems are being caused by the new log file name timestamp

[jnm2]

https://github.com/Microsoft/vstest/pull/1877 is causing problems.

• VSTest inventing directories that don't exist because the string.Replace is hitting all containing directory names, not just the file name.
• The filename is unsightly and there's no way to turn this off.
• Our CI scripts have to guess at the resulting filename based on the directory listing or else direct VSTest to a temp folder and copy TRX files back out with proper filenames.

[mayankbansal018]

@jnm2 we are looking at this issue, & re-designing on how we should handle the results at dotnet sln(multiple test project / multi target test project) level.

We are not reverting the fix done earlier immediately, as it might cause confusion for those whose trx are getting published.

We will publish an RFC once we have a design in place, before implementing it.

[jnm2]

Good to know. I'd love if there was a way for the TRX format to be enhanced so that a single file can contain all the results for each target framework. Projects that use TRX would be able to follow the <multi-project run name>.trx pattern that is possible with NUnit XML.

Second-best would be <multi-project run name>.netstandard2.0.trx etc, but that puts a burden on our CI scripts to play catch-up with our projects' changing TFMs over time. It's not very maintainable from that perspective.

[jcoburn]

well maybe you could use the existing feature? --logger "rx;LogFileName=Framework.trx" for framework

--logger "rx;LogFileName=Core2_1.trx" for Core 2.1

--logger "rx;LogFileName=Core2_2.trx" for Core 2.2

this change broke all of my multi-targeted builds!

[jnm2]

well maybe you could use the existing feature?

How would my build script know which TFMs to use? They change pretty often. This is additional pain.

[singhsarab]

@vagisha-nidhi Can you please link the RFC for this fix and also release when it is available.

[drognanar]

Tagging @AbhitejJohn

[0xced]

#1877 was reverted in #1996 so that fixes the log file name timestamp part of the issue.

Being able to run tests on multiple frameworks and collect the results is unfortunately not yet solved, though.

See also #1876 which is related to this issue. If the [targetFramework] template is implemented as proposed, it could solve the problem of running tests on multiple frameworks by producing one test results file per target framework.

[Evangelink]

Note to self: test if problem still exists

[0xced]

Cross-posting my comment from #1876 which can workaround this issue by defining the test logger parameters as MSBuild properties, where you have access to the target framework and the project name.

I recently learnt that you can control the loggers through MSBuild properties instead of dotnet test arguments. Thus it becomes easy to craft the log file name of your choice. Here's what I have added to the Directory.Build.props file in my tests directory:

<ItemGroup>
  <VSTestLogger Include="trx%3BLogFileName=TestResults-$(TargetFramework)-$(MSBuildProjectName).trx" />
  <VSTestLogger Include="html%3BLogFileName=TestResults-$(TargetFramework)-$(MSBuildProjectName).html" />
</ItemGroup>

<PropertyGroup Condition="$(ContinuousIntegrationBuild) == 'true'">
  <VSTestResultsDirectory>$(MSBuildThisFileDirectory)\..</VSTestResultsDirectory>
  <VSTestLogger>@(VSTestLogger)</VSTestLogger>
</PropertyGroup>