J2N icon indicating copy to clipboard operation
J2N copied to clipboard

Task: Investigate using AzurePipelines.TestLogger for direct test logging via API

Open NightOwl888 opened this issue 6 months ago • 0 comments

Our current test logging is done to a local file on the build server via TRX format, then the TRX file is uploaded to Azure Pipelines using the PublishTestResults@2 task. This works and allows us to add a title per file upload, allowing us to add the test project name, target framework, OS, and build platform on the top level element so we can see where the test failed.

image

This works well, but being that we have so many different test runs and tasks have to run after the tests are completed, it takes a lot of extra time to push the test results (mostly because it runs a bunch of task conditions only to find out that the task is being skipped because it is not the current configuration).

When we were on TeamCity, the test results automatically were added to the job in real time (including counting up the statistics). This allowed us to start investigating a problem as soon as it appeared in the portal instead of waiting for the whole run to finish and the file to upload before seeing anything. This could mean the difference between seeing the problem 5 minutes into the test cycle instead of waiting 20 to 30 minutes.

The AzurePipelines.TestLogger may do the trick, but I don't think it can report the test project name, target framework, OS, and build platform. So, we may need to fork it and see if we can find a way to add that info, and possibly submit a PR to them.

However, first we need to hook it up to see what information it currently provides.

Once we have a solution for this, it can also be used on ICU4N, RandomizedTesting, Spatial4n, and Morfologik.Stemming.

Not sure it can be used on Lucene.NET, though - we would need approval and we currently don't have enough permissions to setup a new Azure DevOps pipeline, much less use a plugin for it. It would work for our "unofficial" pipelines, though.

NightOwl888 avatar Dec 31 '23 07:12 NightOwl888