aqa-test-tools icon indicating copy to clipboard operation
aqa-test-tools copied to clipboard

Optimize Parsing & Storage of Raw Values

Open piyush286 opened this issue 5 years ago • 1 comments

Problem Description

In order to extract Perf results for builds that have multiple tests in them (https://github.com/AdoptOpenJDK/openjdk-test-tools/issues/136), we decided to store raw values in the aggregate info data structure.

The design is discussed: https://github.com/AdoptOpenJDK/openjdk-test-tools/issues/136#issuecomment-582062349

Pros of current design:

  • Makes parser simpler
  • Needs to done only once when the build finishes
  • Different front end tools doesn't require special parsing code.
  • Avoids somewhat duplication of code in the front end since front end tools would require similar logic as the one that's already there in the back end.

Cons of current design:

  • Duplicates raw values by collecting it from different places/builds and putting them in an array

We could re-visit this design to see whether we can make this more efficient to possibly avoid duplicating raw values.

Related PR: https://github.com/AdoptOpenJDK/openjdk-test-tools/pull/179

piyush286 avatar Feb 18 '20 17:02 piyush286

As requested:

We mainly have 2 scenarios for running perf tests. Here's a short summary:

Scenario Description Uses Aggregate Data Example
Using Openjdk-tests Framework (TKG) For each JDK, we run multiple tests multiple times in one Jenkins Build. We do the same for the baseline JDK in another pipeline. Yes Test_perf/job/Test_openjdk11_j9_sanity.perf_ppc64le_linux_siena/ ITERATIONS=4 BUILD_LIST=perf/quarkus TARGET=sanity.perf
PerfNext It uses 3 levels: Grandparent, Parent, and Child. For every test, one grandparent job is launched. If interleave is selected, then 2 parent builds are launched by grandparent. Parent build launches multiple child builds, depending on the number of iterations for that specific test. We monitor the parent build. Yes Test_perf/job/PerfNext-Parent

All perf tools (Tabular View, Perf Compare, & Perf Graphs) use aggregate data structure as well.

Please let me know if you need more clarification on something.

piyush286 avatar May 01 '20 21:05 piyush286