EPIC: Ensure test material is pinned to a specific tag and SHAs tracked

[smlambert]

There are several different repositories where we pull test material whose SHAs are not tracked in the TAP file, these include:

• [x] Track perf benchmarks - https://github.com/adoptium/aqa-tests/issues/5818
• [x] Track external test applications https://github.com/adoptium/aqa-tests/issues/5825
• [x] #5769 - https://github.com/adoptium/aqa-tests/issues/5818

This refers mainly to material pulled in via ant scripts (build.xml files). Any of the test material included in the testenv.properties file gets written to the TAP files.

In addition to tracking test material, we should also check that any dependencies introduced are also tracked / verified against a checksum to ensure it is transparent and clear what is being put onto test machines.

Dependencies (any software that is pulled onto the machine during the test run) include:

• prereqs that the Ansible playbooks install / or are defined in Dockerfiles

• dependencies pulled in via getDependency jobs

• [ ] https://github.com/adoptium/aqa-tests/issues/5954

  • These are scripts outside of the mechanisms listed above (i.e. Ansible playbooks, getDependency list), without verification for a checksum, and especially if they are from unofficial or personal branches. We should discourage the use of such scripts as it introduces a level of insecurity that we want to move away from.

[smlambert]

Likely related:

• https://github.com/adoptium/aqa-tests/issues/5288
• https://github.com/adoptium/aqa-tests/issues/5151