aqa-test-tools
aqa-test-tools copied to clipboard
Use Common Benchmark Metric Router Design for All Perf Tools
Problem Description
Currently, TRSS has 2 parsing mechanisms for extract benchmark metric values.
First mechanism is used for Jenkins, which is used by PerfNext and Adopt builds, to parse and store results automatically when a build finishes.
Second mechanism is used for Axxon schedular, which is used by Perffarm currently, to extract numbers from a CSV results file that's generated by Perffarm. Some files related to this mechanism are also used in all frontend perf tools such as Tabular View and Perf Compare.
Maintaining 2 mechanism adds too much redundancy, making it harder to maintain and support different tools.
Proposed Changes
- Unify both the parsing mechanisms.
- Add
higherbetterandunitsto BenchmarkMetric. Currently, BenchmarkMetric doesn't have that information and we rely on BenchmarkVariants for it. - Get rid of BenchmarkVariants file.
- Make an API so that all perf tools can request the backend for the BenchmarkMetric file and use it accordingly.
- Update perf tools to use that API to fetch BenchmarkMetric file instead of using BenchmarkVariants file.
Assigned Contributors
I'll be working with Dong (@dhlee49) from my team to work on this design.