Add test results as parameters

[hunhejj]

Hi, I would like to use this plugin to post the test results after a successful build. Either a summary, e.g. number of

• successful tests: 561
• failed tests: 13
• ignored tests: 64

Or even better beside of the above the detailed test results, e.g. an array of objects, where objects contain the result of a single a test. An object could look something like this:

• test name: TestThatThisIfNotThat
• result: <successful / failed / ignored>
• duration: 250ms
• investigator: John Doe
• flaky flag set by TeamCity: <yes / no>
• first failure: <some_date>

[netwolfuk]

This is a very specific use case, and I would therefore be hesitant to add it to the standard webhook payload. However, it's the sort of thing that would be quite possible once #103 is completed. With a velocity template, one can call methods on an object, and add conditionals and other logic into a webhook payload.

A very quick look at the API looks like I could make the BuildStatistics object part of the Velocity Context and then you could do something like:

successful tests: $buildStatistics.passedTestCount
failed tests: $buildStatistics.failedTestCount
ignored tests: $buildStatistics.ignoredTestCount
muted tests: $buildStatistics.mutedTestsCount

And the more involved case:

#foreach ($test in $buildStatistics.allTests)
    #if( $test.status.isFailed() )
       //do some stuff here with $test (STestRun) object, eg.
       $test.getFullText()
   #end
#end

[hunhejj]

That would be really nice!

[netwolfuk]

I have a proof of concept working using the build from: this branch

#set ($buildStatistics = $build.fullStatistics)
{
    "buildName": "${buildType.name}",
    "projectName" : "${project.externalId}",
    "successfulTests": $buildStatistics.passedTestCount, 
    "failedTests" : $buildStatistics.failedTestCount,
    "ignoredTests": $buildStatistics.ignoredTestCount,
    "mutedTests"  : $buildStatistics.mutedTestsCount,
#foreach ($test in $buildStatistics.allTests)
    #if( $test.status.isIgnored() )
       "test-$test.testRunId" : "$test.test.name - $test.status",
    #end
#end 
    "buildResult" : "${buildResult}"
}

Produces the following:

{
  "buildName": "tcWebHooks - GitHub",
  "projectName": "TcPlugins",
  "successfulTests": 338,
  "failedTests": 0,
  "ignoredTests": 9,
  "mutedTests": 0,
  "test-1640": "webhook.teamcity.reporting.ReportPayloadFactoryImplTest.testBuildReportPayload - UNKNOWN",
  "test-5617": "webhook.teamcity.WebHookContentBuilderTest.testBuildWebHookContent - UNKNOWN",
  "test-6066": "webhook.teamcity.WebHookListenerTest.testBuildChangedStatusSRunningBuildStatusStatus - UNKNOWN",
  "test-6069": "webhook.teamcity.WebHookListenerTest.testBuildStartedSRunningBuild - UNKNOWN",
  "test-6072": "webhook.teamcity.WebHookListenerTest.testBeforeBuildFinishSRunningBuild - UNKNOWN",
  "test-6077": "webhook.teamcity.WebHookListenerTest.testBuildInterruptedSRunningBuild - UNKNOWN",
  "test-6082": "webhook.teamcity.WebHookListenerTest.testBuildFinishedSRunningBuildSuccessAfterSuccess - UNKNOWN",
  "test-6085": "webhook.teamcity.WebHookListenerTest.testBuildFinishedSRunningBuild - UNKNOWN",
  "test-6088": "webhook.teamcity.WebHookListenerTest.testBuildFinishedSRunningBuildSuccessAfterFailure - UNKNOWN",
  "buildResult": "success"
}

[netwolfuk]

Depends on #31

[netwolfuk]

This is available for testing in the v1.2.0-alpha.3 release.