Incorrect Test Targets Displayed for single extended.perf run

[adamfarley]

Summary

In a deeply unusual extended.perf run, several failed test targets are identified by TRSS as "disabled", and several more are identified as "failed" when they actually passed.

https://trss.adoptopenjdk.net/allTestsInfo?buildId=60735c4947db975731f280c2&limit=5&hasChildren=false

Details

This job is unusual in several ways, as described here.

In short, every time a test target fails:

• The message at the end of the test says, incorrectly, that it's a different, earlier target that failed.
• The progression through the list of test targets is somehow reset to that of the earlier test target.
• The next targets to be run will be the ones that are the correct successors to the earlier target.

As a result, dismissing this bug as GIGO would be fair, though perhaps TRSS could be hardened to these sorts of shenanigans?

Also note that this issue has only been seen once so far, across all versions, platforms, and VMs (OpenJ9). See the repro link below for an attempt to reproduce.

Repro Links https://ci.adoptopenjdk.net/job/Grinder/102/ Result: Saw one StackOverflow, but we need more than one to reproduce this.

Bigger repro: https://ci.adoptopenjdk.net/job/Grinder/114/

Grinder Link https://ci.adoptopenjdk.net/job/Grinder/parambuild/?SDK_RESOURCE=upstream&TARGET=extended.perf&TEST_FLAG=&UPSTREAM_TEST_JOB_NAME=&DOCKER_REQUIRED=false&ACTIVE_NODE_TIMEOUT=0&VENDOR_TEST_DIRS=&EXTRA_DOCKER_ARGS=&TKG_OWNER_BRANCH=AdoptOpenJDK%3Amaster&TEST_PARALLELIZATION_PARAMS=&PLATFORM=ppc64_aix&KEEP_REPORTDIR=false&PERSONAL_BUILD=false&ADOPTOPENJDK_REPO=https%3A%2F%2Fgithub.com%2FAdoptOpenJDK%2Fopenjdk-tests.git&LABEL=&TEST_OPTIONS_PARAMS=&EXTRA_OPTIONS=&CUSTOMIZED_SDK_URL=&BUILD_IDENTIFIER=&NON_AQA_TEST_REPOS_HELP_TEXT=&ADOPTOPENJDK_BRANCH=master&LIGHT_WEIGHT_CHECKOUT=false&NON_AQA_TEST_REPOS=&ARTIFACTORY_SERVER=&TEST_REPO_PARAMS=&TEST_SELECTION_PARAMS=&TEST_PARALLELIZATION_PARAMS_HELP_TEXT=&JDK_SELECTION_PARAMS=&KEEP_WORKSPACE=false&USER_CREDENTIALS_ID=&JDK_VERSION=11&ITERATIONS=1&VENDOR_TEST_REPOS=&JDK_REPO=https%3A%2F%2Fgithub.com%2Fibmruntimes%2Fopenj9-openjdk-jdk11&PLATFORM_AND_MACHINE_HELP_TEXT=&RELEASE_TAG=&OPENJ9_BRANCH=master&OPENJ9_SHA=&JCK_GIT_REPO=&VENDOR_TEST_BRANCHES=&UPSTREAM_JOB_NAME=build-scripts%2Fjobs%2Fjdk11u%2Fjdk11u-aix-ppc64-openj9&OPENJ9_REPO=https%3A%2F%2Fgithub.com%2Feclipse%2Fopenj9.git&PLATFORM_AND_MACHINE=&CUSTOM_TARGET=&VENDOR_TEST_SHAS=&JDK_BRANCH=openj9&LABEL_ADDITION=&ARTIFACTORY_REPO=&ARTIFACTORY_ROOT_DIR=&POST_RUN_PARAMS_HELP_TEXT=&UPSTREAM_TEST_JOB_NUMBER=&DOCKERIMAGE_TAG=&JDK_SELECTION_PARAMS_HELP_TEXT=&JDK_IMPL=openj9&SSH_AGENT_CREDENTIAL=&AUTO_DETECT=true&TKG_SHA=&TEST_SELECTION_PARAMS_HELP_TEXT=&CUSTOMIZED_SDK_URL_CREDENTIAL_ID=&OPENJDK_SHA=&NUM_MACHINES=&BUILD_LIST=perf&UPSTREAM_JOB_NUMBER=964&TEST_REPO_PARAMS_HELP_TEXT=&POST_RUN_PARAMS=&TIME_LIMIT=10&TEST_OPTIONS_PARAMS_HELP_TEXT=&JVM_OPTIONS=&PARALLEL=None