17.11 - error output is treated as error and fails tests

[birarroshan]

ℹ️ workarounds

Set ForwardStandardOutput to false in your runsettings:

<RunSettings>
  <RunConfiguration>
    <ForwardStandardOutput>false</ForwardStandardOutput>
  </RunConfiguration>
</RunSettings>

or

set VSTEST_DISABLE_STANDARD_OUTPUT_FORWARDING=1 environment variable.

---

Description

We are running Appium-WinAppdriver UI tests with ADO pipeline. After upgrading to 17.11.0, VStest task is reporting appium debug logs as error.

Steps to reproduce

1. Install 17.11.0 for VSTestPlatform - Latest Stable is 17.11.0

• task: VisualStudioTestPlatformInstaller@1 inputs: packageFeedSelector: 'nugetOrg' versionSelector: 'latestStable'

2. Run test using
   • task: VSTest@2 inputs: testSelector: 'testAssemblies' testAssemblyVer2: "Test.dll" searchFolder: $(Pipeline.Workspace) testFiltercriteria: "$(testFilterExpression)" vsTestVersion: 'toolsInstaller'

Expected behavior

Tests should run as usual without any error logs

Actual behavior

Appium DEBUG logs are being reported as Error. Note - We have not enabled any debug logging for appium

Also, specFlow logs are reported as Warning.

Diagnostic logs

Could not collect Diagnostic logs. Will upload it one collected.

Environment

OS - Hosted Pool with windows 10 base image

Aditional Notes -

ADO job is passing, but all errors are reported under Error tab of pipeline run