TestLogger/DataCollector can't be used to log Test Case Start

[davidmatson]

trafficstars

Description

I'm trying to use VSTest extensibility to integrate with another logging system that needs to report as soon as a test case starts. ITestLogger allows me to log on TestCaseEnd, but it doesn't provide any indication when a test case starts. This functionality is important especially for long-running test cases.

I tried using DataCollector and InProcessDataCollection, but neither of those help either - neither receive other messages as a test is running, unlike ITestLogger. Even using all in combination, I still can't get all the events as they happen - InProcessDataCollection doesn't have a way to log messages out; DataCollector can only log warnings or errors, but even these aren't sent to the ITestLogger right away - the platform appears to buffer any logged messages after the first one, so sending a "test case start" message from a DataCollector to an ITestLogger also does not work.

This scenario might be possible by replacing vstest.console.exe entirely with my own code, but that's far more code than I'd like to own just to be notified of and log TestStart events.

This is a feature request to add a TestStart event to TestLoggerEvents.

[nohwnd]

Hello, are you using some specific testing framework? Or this is a for a "service" and needs to take all testing frameworks into account?

[davidmatson]

This is a for a service and needs to take all testing frameworks into account.

[reinhardsaller]

I would also be interested in this, the use case being the attribution of received messages to the test cases that sent them via IFrameworkHandle.SendMessage(). This might require to forward events to the test loggers unbuffered, though, and I don't think that's the case at the moment.

@davidmatson, when you say

ITestLogger allows me to log on TestCaseEnd

are you referring to the TestLoggerEvents.TestResult event?

[davidmatson]

@davidmatson, when you say ITestLogger allows me to log on TestCaseEnd are you referring to the TestLoggerEvents.TestResult event?

Yes. I was originally thinking of how test framework adapters call RecordEnd but RecordResult is probably more accurate.

[nohwnd]

This is a new feature and won't be implemented, we are focusing on adding new features to Testing.Platform instead. https://aka.ms/testingplatform