Streaming output doesn't provide test part result type (can't distinguish success message vs skipped vs failure message)

[davidmatson]

Describe the bug

When processing streaming output for TestPartResult, it's not possible to tell if it's an error or not.

Steps to reproduce the bug

TEST(SomeFixture, SampleExplicitSucceed)
{
    SUCCEED();
}

TEST(SomeFixture, SampleFails)
{
    constexpr int x{ 1 };
    constexpr int y{ 2 };
    ASSERT_EQ(x, y);
}

Does the bug persist in the most recent commit?

yes

What operating system and version are you using?

Windows 10 (10.0.19044.1889)

What compiler and version are you using?

MSVC with VS 2022 (VS 17.3.3)

What build system are you using?

Visual Studio Additional context

Add any other context about the problem here.

The streaming output looks like this:

...
event=TestPartResult&file=path\to\some_file.cpp&line=15&message=Succeeded
...
event=TestPartResult&file=path\to\some_file.cpp&line=22&message=Expected equality of these values:%0A  x%0A    Which is: 1%0A  y%0A    Which is: 2
...

The TestPartResult contains a type field - if that were preserved in event=TestPartResult (&type=...), programmatic marking of the message as error or not would be possible.

[davidmatson]

Even worse:

#include <gtest/gtest.h>

TEST(SampleFixture, SuccessWithCustomMessage)
{
    GTEST_SUCCEED() << "custom message";
}

TEST(SampleFixture, SkipWithCustomMessage)
{
    GTEST_SKIP() << "custom message";
}

TEST(SampleFixture, FailWithCustomMessage)
{
    GTEST_FAIL() << "custom message";
}

TEST(SampleFixture, FailNonFatalWithCustomMessage)
{
    ADD_FAILURE() << "custom message";
}

Produces this output:

...
event=TestPartResult&file=path\to\some_file.cpp&line=5&message=Succeeded%0Acustom message
...
event=TestPartResult&file=path\to\some_file.cpp&line=10&message=custom message
...
event=TestPartResult&file=path\to\some_file.cpp&line=15&message=Failed%0Acustom message
...
event=TestPartResult&file=path\to\some_file.cpp&line=20&message=Failed%0Acustom message
...

(there isn't a good way to determine programmatically that the skipped one ended as Skipped or fatal vs non-fatal failure; even Succeeded vs Failed aren't great)

[davidmatson]

Proposed fix: Change the event=TestPartResult... to add &type=. For example:

event=TestPartResult&file=path\to\some_file.cpp&line=123&type=success&message=custom message
event=TestPartResult&file=path\to\some_file.cpp&line=123&type=failure&message=custom message
event=TestPartResult&file=path\to\some_file.cpp&line=123&type=non_fatal_failure&message=custom message
event=TestPartResult&file=path\to\some_file.cpp&line=123&type=skip&message=custom message

[asoffer]

Can you say a bit more? What do you mean by "processing"? By "streaming" do you mean text written to stderr and stdout?

[davidmatson]

Can you say a bit more? What do you mean by "processing"? By "streaming" do you mean text written to stderr and stdout?

Sorry, I should have clarified - by "streaming" I meant --gtest_stream_result_to. (By "processing" I mean just having access to that data in what's written to the --gtest_stream_result_to socket.)

[davidmatson]

#4126