Release 1.30 all test results are duplicated

[oosterholt]

Since version 1.30 (1.29 still works), all my tests are reported twice, duplicating all test results. Probably caused by fix JENKINS-48583.

Each suite is reported once, all tests inside it are reported twice (once with the original name and once appended with '_2'; e.g. my.package.name.suite.testCaseOne and my.package.name.suite.testCaseOne_2).

Configured publish report 'fileset includes': **/report/test/TEST-*.xml

I'm also using the Parallel Test Executor plugin to run all my tests in parallel in 6 jobs running on different nodes.

If more information is required please let me know.

Also posted this bug in Jenkins buglist (not sure where to file this issue; was no response in Jenkins ticket).

[timja]

Do you have an example project this can be with reproduced with? / can you provide one?

and what type of project are you using? maven, freestyle, pipeline?

[oosterholt]

Do you have an example project this can be with reproduced with? / can you provide one?

I'm trying to create a demo situation for reproduction. I'll let you know. It has definitely something to do with the Parallel Test Executor plugin.

what type of project are you using? maven, freestyle, pipeline?

I'm using a freestyle project.

[oosterholt]

Here are the test report files in the 'upstream job' (see parallel-test-executor).

reports.zip

[timja]

If you could bisect what change introduced this it would help otherwise I’ll try to get to it in the next couple of weeks

[oosterholt]

@timja What is the best way of doing this. When I clone this repo I do not even see the 'src' directory for example.

[timja]

if you run mvn package -P quick-build you will get an hpi that you can load into your Jenkins.

or you can run: mvn hpi:run -P quick-build which will start up a Jenkins server based on your commit

[oosterholt]

Cool thanks. That's fast. Since I do not see the sources, how can I bisect?

[timja]

is this in your IDE you don't see them? when you checkout the code it's in the src folder

[oosterholt]

Revision 3a90162733384353be4ca8ef138c3f7a63fecc51 is causing this issue.

[timja]

Thanks and do you have an example project that you can reproduce this with? and a pipeline?

(it'll make it much easier for me or others to look at a fix if we have that)

[rpjesivac]

@timja @oosterholt Any progress on this? We have upgraded to the 1.43 version and now we are seeing this issue as well.

[oosterholt]

@rpjesivac do you have the same job setup; are you also using the Parallel Test Executor plugin?

[zbynek]

@rpjesivac in your folder all the XMLs are duplicated: see reports\2\report\test\TEST-our.package.activiti.connector.ActivitiPTBConnectorTest.xml vs reports\2\report\test\TEST-out.package.activiti.connector.ActivitiPTBConnectorTest.xml

Any idea what causes that?

Also, the following pipeline downloads your test results and runs them through the plugin, but I couldn't reproduce the bug:

pipeline {
    agent any
    stages {
        stage('Example') {
            steps {
            sh '''
            wget https://github.com/jenkinsci/junit-plugin/files/5318011/reports.zip
            rm -rf reports
            unzip reports.zip
            find reports | sed 's/.*/sleep .1 \\&\\& touch &/' | sh
            '''
            junit '**/test/*.xml'
            }
        }
    }
}

[oosterholt]

The 'Test-out.*' files are faulty. Files got duplicated while 'obfuscating' test reports. I've updated the reports.zip with the duplicated files removed. These are not the reason of the duplicated test results.

[oosterholt]

Still no clue what is going on. I've added a archive artefacts post build step with the same pattern as the publish test result. The artefacts do not show any duplicated test report files. A text search on any of the duplicate reported test only results in one match.

There are no changes in the archived test report file artefacts in a test run with the 1.29 version of the plugin and a test run with any of the newer plugin versions.

[oosterholt]

Additional information to clarify the situation. All tests are reported twice; see test result chart: Also see any of the test suites; all test methods are reported twice:

All test are reported once with their normal name: ...lastCompletedBuild/testReport/our.package.activiti/DocumentApprovalQuickTest/cancelWorkflow and once with a nummer appended: ...lastCompletedBuild/testReport/our.package.activiti/DocumentApprovalQuickTest/cancelWorkflow_2